Failed to start Zabbix Agent

[Mammosta32]

I am starting to get very frustrated with Zabbix because of the agent service will not start. When I try to run systemctl start zabbix-agent I get the following error below. I would greatly appreciate if someone can tell me what am I doing wrong or what causes this. Zabbix server is running as well as the DB is active, zabbix agent is the only thing I can't get going.

[root@zbx01 zabbix-server-mysql-3.0.8]# systemctl status zabbix-agent.service
● zabbix-agent.service - Zabbix Agent
Loaded: loaded (/usr/lib/systemd/system/zabbix-agent.service; enabled; vendor preset: disabled)
Active: activating (auto-restart) (Result: resources) since Fri 2017-05-05 16:04:31 EDT; 4s ago
Process: 1943 ExecStop=/bin/kill -SIGTERM $MAINPID (code=exited, status=1/FAILURE)
Process: 9487 ExecStart=/usr/sbin/zabbix_agentd -c $CONFFILE (code=exited, status=0/SUCCESS)
Main PID: 1942 (code=exited, status=1/FAILURE)

May 05 16:04:31 zbx01.centos7 systemd[1]: zabbix-agent.service never wrote i....
May 05 16:04:31 zbx01.centos7 systemd[1]: Failed to start Zabbix Agent.
May 05 16:04:31 zbx01.centos7 systemd[1]: Unit zabbix-agent.service entered ....
May 05 16:04:31 zbx01.centos7 systemd[1]: zabbix-agent.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

[kloczek]

So what is in the agent log file?

[Mammosta32]

Unable to start zabbix agent

10789:20170505:162337.735 **************************
10789:20170505:162337.735 using configuration file: /etc/zabbix/zabbix_agentd.conf
10789:20170505:162337.735 cannot set resource limit: [13] Permission denied
10789:20170505:162337.735 cannot disable core dump, exiting...
10797:20170505:162341.801 Starting Zabbix Agent [Zabbix server]. Zabbix 3.0.8 (revision 65976).
10797:20170505:162341.801 **** Enabled features ****
10797:20170505:162341.801 IPv6 support: YES
10797:20170505:162341.801 TLS support: YES
10797:20170505:162341.801 **************************
10797:20170505:162341.801 using configuration file: /etc/zabbix/zabbix_agentd.conf
10797:20170505:162341.803 agent #0 started [main process]
10802:20170505:162341.804 agent #5 started [active checks #1]
10798:20170505:162341.806 agent #1 started [collector]
10799:20170505:162341.806 agent #2 started[listener #1]
10800:20170505:162341.808 agent #3 started[listener #2]
10801:20170505:162341.809 agent #4 started[listener #3]
zabbix_agentd [10806]: Is this process already running? Could not lock PID file [/var/run/zabbix/zabbix_agentd.pid]: [11] Resource temporarily unavailable

[Mammosta32]

Unable to start zabbix agent

So what is in the agent log file?

10789:20170505:162337.735 **************************
10789:20170505:162337.735 using configuration file: /etc/zabbix/zabbix_agentd.conf
10789:20170505:162337.735 cannot set resource limit: [13] Permission denied
10789:20170505:162337.735 cannot disable core dump, exiting...
10797:20170505:162341.801 Starting Zabbix Agent [Zabbix server]. Zabbix 3.0.8 (revision 65976).
10797:20170505:162341.801 **** Enabled features ****
10797:20170505:162341.801 IPv6 support: YES
10797:20170505:162341.801 TLS support: YES
10797:20170505:162341.801 **************************
10797:20170505:162341.801 using configuration file: /etc/zabbix/zabbix_agentd.conf
10797:20170505:162341.803 agent #0 started [main process]
10802:20170505:162341.804 agent #5 started [active checks #1]
10798:20170505:162341.806 agent #1 started [collector]
10799:20170505:162341.806 agent #2 started[listener #1]
10800:20170505:162341.808 agent #3 started[listener #2]
10801:20170505:162341.809 agent #4 started[listener #3]
zabbix_agentd [10806]: Is this process already running? Could not lock PID file [/var/run/zabbix/zabbix_agentd.pid]: [11] Resource temporarily unavailable

[Atsushi]

Have you enabled SELinux?
Please allow permission properly or try disabling it.

[kloczek]

zabbix_agentd [10806]: Is this process already running? Could not lock PID file [/var/run/zabbix/zabbix_agentd.pid]: [11] Resource temporarily unavailable

And /var/run/zabbix exist, and/or is created as temp directory by systemd, and is owned by zabbix:zabbix?

[Erik.Raaijmakers]

Hi,

If this can be of any help: with the following package version of the zabbix agent,
I'm experiencing the same problem on every server where I install it for the first time:

zabbix-agent 1:3.2.6-1+xenial amd64 "Zabbix network monitoring solution - agent"

At initial installation of the package the agent starts. However, after server reboot, the agent fails to start (the service is enabled in systemd).
This because the tmpfs /var/run/zabbix directory has not been created by systemd, and so the agent cannot store its PID in that expected directory.

When I execute manually the command that should have created this directory at reboot (systemd-tmpfiles --create /usr/lib/tmpfiles.d/zabbix-agent.conf),
then /var/run/zabbix is properly created, owned by zabbix:zabbix, and the zabbix agent starts again.

Contents of usr/lib/tmpfiles.d/zabbix-agent.conf, cat /usr/lib/tmpfiles.d/zabbix-agent.conf:

d /var/run/zabbix 0755 zabbix zabbix - -

Syslog shows the reason why /var/run/zabbix was not created when booting the server:

Jun 1 13:29:21 <server_hostname> systemd-tmpfiles[867]: [/usr/lib/tmpfiles.d/zabbix-agent.conf:1] Unknown user 'zabbix'.

So it looks like user zabbix does not exist yet on that moment in the boot sequence? I did not experience this problem on previous versions of the package.