Zabbix Agent not starting on boot - RHEL 5.9

[StarDestroyer]

I have a RHEL 5.9 server that I installed Zabbix Agent on using the RPM repository (current package is zabbix-agent-2.2.4-1.el5.x86_64). For various reasons, this server reboots nightly. For some reason, the Zabbix Agent does not start at boot time. If I start it from the command line, it starts up with no issues.

#chkconfig --list zabbix-agent
zabbix-agent    0:off   1:off   2:on    3:on    4:on    5:on    6:off
#tail /var/log/zabbix/zabbix_agentd.log
   899:20140707:153041.448 agent #0 started [collector]
   900:20140707:153041.448 agent #1 started[listener #1]
   901:20140707:153041.448 agent #2 started[listener #2]
   902:20140707:153041.449 agent #3 started[listener #3]
   899:20140708:022207.083 Got signal [signal:15(SIGTERM),sender_pid:587,sender_uid:0,reason:0]. Exiting ...
   897:20140708:022207.083 Got signal [signal:15(SIGTERM),sender_pid:587,sender_uid:0,reason:0]. Exiting ...
   900:20140708:022207.083 Got signal [signal:15(SIGTERM),sender_pid:587,sender_uid:0,reason:0]. Exiting ...
   902:20140708:022207.083 Got signal [signal:15(SIGTERM),sender_pid:587,sender_uid:0,reason:0]. Exiting ...
   901:20140708:022207.083 Got signal [signal:15(SIGTERM),sender_pid:587,sender_uid:0,reason:0]. Exiting ...
   897:20140708:022207.539 Zabbix Agent stopped. Zabbix 2.2.4 (revision 46772).
#grep zabbix /var/log/messages
#grep zabbix /var/log/dmesg
#service zabbix-agent start
Starting Zabbix agent:                                     [  OK  ]
#tail /var/log/zabbix/zabbix_agentd.log
   900:20140708:022207.083 Got signal [signal:15(SIGTERM),sender_pid:587,sender_uid:0,reason:0]. Exiting ...
   902:20140708:022207.083 Got signal [signal:15(SIGTERM),sender_pid:587,sender_uid:0,reason:0]. Exiting ...
   901:20140708:022207.083 Got signal [signal:15(SIGTERM),sender_pid:587,sender_uid:0,reason:0]. Exiting ...
   897:20140708:022207.539 Zabbix Agent stopped. Zabbix 2.2.4 (revision 46772).
 11974:20140708:101002.264 Starting Zabbix Agent [qad.hilite-ind.net]. Zabbix 2.2.4 (revision 46772).
 11974:20140708:101002.264 using configuration file: /etc/zabbix/zabbix_agentd.conf
 11975:20140708:101002.265 agent #0 started [collector]
 11976:20140708:101002.266 agent #1 started[listener #1]
 11977:20140708:101002.266 agent #2 started[listener #2]
 11978:20140708:101002.267 agent #3 started[listener #3]

I'm familiar with Linux, but not quite as familiar as I'd like to be. I can't think of anywhere else to look. This is a production server for our ERP system. While it does reboot nightly, I can't just reboot at will in order to troubleshoot this.

[desmondho]

Sound like the chkconfig having problem, please check with the following link to see for more information

http://serverfault.com/questions/343...ript-on-bootup

[StarDestroyer]

Thanks for the reply... The linked article mentions two possible issues...

1. Things starting in the wrong order.

That does not seem to be the case here as the symlinks are properly created as "S86zabbix-agent" and "K14zabbix-agent".

2. Path issues in the init script.

The link doesn't really expand on that suggestion as their problem was that the init script had the wrong syntax for defining the start order (problem #1).

This is the only RHEL server that I have, but I do have a CentOS v5 server. It appears that everything is working fine on that machine for loading the Zabbix Agent on boot.

[StarDestroyer]

I found that RHEL 5.3+ seems to have a bug that causes boot.log to not populate properly. I found a solution for that problem (https://access.redhat.com/solutions/9834 - requires RHEL support login) and applied that yesterday. Today, I checked the boot log.

#grep zabbix /var/log/boot.log
Starting Zabbix agent: zabbix_agentd [4840]: user zabbix does not exist
zabbix_agentd [4840]: cannot run as root!

This particular server is configured to allow both local logins and logins using Active Directory credentials. It does look like the winbind process starts later on in the process (/etc/rc.d/rc3.d/S91winbind). I would think that the zabbix user would have been a local user and not depend on the winbind service, but perhaps not.