Zabbix Server Not Running - Cannot Restart - Unable to write pid

[BSDUKJ]

I'm Running Zabbix 3.4.5 on Centos 7.4.1708 and have been following thread: https://www.zabbix.com/forum/showthread.php?t=59895 but it's not quite the same as I have had Zabbix running fine for over 6 months the zabbix server has just stopped running.

I believe I caused the issue as I configured 3 Actions related to 3 Discoveries on different subnets at the same time and I don't think my zabbix configuration has been able to handle it.

I have clicked to disable all of them but it hasn't worked, please see the results of the restart zabbix-server.service and status below:

[root@zabbixsvr-domain user]# systemctl restart zabbix-server.service
Job for zabbix-server.service failed because a configured resource limit was exceeded. See "systemctl status zabbix-server.service" and "journalctl -xe" for details.
[root@zabbixsvr-domain user]# systemctl status zabbix-server.service -l
● zabbix-server.service - Zabbix Server
Loaded: loaded (/usr/lib/systemd/system/zabbix-server.service; enabled; vendor preset: disabled)
Active: activating (auto-restart) (Result: resources) since Sat 2017-12-30 02:18:34 GMT; 7s ago
Process: 5586 ExecStop=/bin/kill -SIGTERM $MAINPID (code=exited, status=1/FAILURE)
Process: 5611 ExecStart=/usr/sbin/zabbix_server -c $CONFFILE (code=exited, status=0/SUCCESS)
Main PID: 5570 (code=exited, status=1/FAILURE)

Dec 30 02:18:34 zabbixsvr-domain systemd[1]: zabbix-server.service never wrote its PID file. Failing.
Dec 30 02:18:34 zabbixsvr-domain systemd[1]: Failed to start Zabbix Server.
Dec 30 02:18:34 zabbixsvr-domain systemd[1]: Unit zabbix-server.service entered failed state.
Dec 30 02:18:34 zabbixsvr-domain systemd[1]: zabbix-server.service failed.

I hope someone can help me or point me in the right direction.

Cheers

J

[LenR]

Is selinux disabled?

[kloczek]

Is selinux disabled?

or added correct SELinux policies ..

[BSDUKJ]

There were firewall rules initially put in place however I disabled selinux prior to adding devices to zabbix

it was disabled 6 months ago.

When I view the status of selinux is displays disabled.

The only real change I ses is that I overloaded it with discoveries.

I kept the default settings in the configuration for everything.

I've tried uping the cache and start pollers and have cleared 10GB of space on the partition.

The selinux is interesting as I've had similar firewall issues with Micro$oft Windows when disabled but they still appear follow the rules, could it be something similar here?

[BSDUKJ]

I've looked at all the policies and everything appears fine.

It does look like a permissions thing, but I don't know what to look for.

[jampei]

I've looked at all the policies and everything appears fine.

It does look like a permissions thing, but I don't know what to look for.

check the zabbix logs and paste

[BSDUKJ]

I've managed to get the zabbix server up and running, using this link:

https://www.1and1.com/cloud-community/learn/database/mysqlmariadb/solve-a-mysqlmariadb-too-many-connections-error/

However it was short lived and has failed again.

[BSDUKJ]

Finally got this resolved!

in my.cnf, under [mysqld] I added following lines:

max_connections=500
innodb_buffer_pool_size=1G

Also in zabbix_server.conf I changed to the following values:

StartPollers=3 to StartPollers=50
StartPollersUnreachable=1 to StartPollers=50
StartPingers=3 to StartPingers=10
CacheSize=8M to CacheSize=2G
Timeout=3 to Timeout=15

Restarted MySQL and Zabbix service.

[LenR]

I suspect you were getting error messages in the zabbix server log that weren't in the post or the log level was too low to see them. If it was the mysql connection limit, those were pretty clear messages when I've seen them in the past.

Finding the point in the logs where messages go from good to bad can be several 100 lines back up :-)

Glad you got it going, we have been using Zabbix since around 1.8 and it has saved us many wake up calls in the middle of the night :-)