Zabbix server dies when I start the agent

[guille_pm]

Hi everyone! I'm new to Zabbix and Linux, so I apologize for any mistake/confusion I may cause.
I have installed Zabbix server and agent on the same system, was able to configure and get the server running, but as soon as I start the agent, the server dies. My config is
PowerPC arch
Centos 7
Installed from sources

Server log says:

3486:20200403:103617.883 One child process died (PID:3530,exitcode/signal:1). Exiting ...
zabbix_server [3486]: Error waiting for process with PID 3530: [10] No child processes
3486:20200403:103617.894 syncing history data...
3486:20200403:103617.903 syncing history data... 100.000000%
3486:20200403:103617.903 syncing history data done
3486:20200403:103617.903 syncing trend data...
3486:20200403:103617.911 syncing trend data done
3486:20200403:103617.911 Zabbix Server stopped. Zabbix 4.4.7 (revision 77fb8c7ee0).


I have made a couple of test, the processes mentioned in the log are these
First test
zabbix_server: preprocessing worker #1
Second test
zabbix_server: preprocessing worker #3

Agent doesn't show errors

4502:20200403:103601.260 Starting Zabbix Agent [Zabbix server]. Zabbix 4.4.7 (revision 77fb8c7ee0).
4502:20200403:103601.260 **** Enabled features ****
4502:20200403:103601.260 IPv6 support: NO
4502:20200403:103601.260 TLS support: NO
4502:20200403:103601.260 **************************
4502:20200403:103601.260 using configuration file: /usr/local/etc/zabbix_agentd.conf
4502:20200403:103601.260 agent #0 started [main process]
4503:20200403:103601.260 agent #1 started [collector]
4504:20200403:103601.261 agent #2 started[listener #1]
4505:20200403:103601.261 agent #3 started[listener #2]
4506:20200403:103601.261 agent #4 started[listener #3]
4507:20200403:103601.261 agent #5 started [active checks #1]
4508:20200403:103601.261 agent #6 started [active checks #2]


I made a search, but could not find anything that looked like a match...

Thanks!

[tim.mooney]

That's very strange.

Can you increase the log level for your zabbix_server (see the man page for zabbix_server), stop both server (even though it's not running) and agent, and then start the server, wait a little bit, and start the agent? I'm interested in whether the server logs any additional information.

Might be worth checking the system logs for timestamps around the same time that the server dies, to see if anything else gets logged.

I've not run zabbix on CentOS on PowerPC, are there any resource limits in place that may be interfering with one user (the zabbix user) running multiple processes?

[guille_pm]

I did that, here are the server and agent logs. I'm sorry I had to split the server log, but it wouldn't allow me to upload it. Just remove the last .zip to decompress it.
The /var/log/messages has the following message, but none match with the timestamp of the problem

Apr 3 10:25:46 zbxll01 systemd: Reloading.
Apr 3 11:01:01 zbxll01 systemd: Started Session 3 of user root.
Apr 3 12:01:01 zbxll01 systemd: Started Session 4 of user root.
Apr 3 13:01:01 zbxll01 systemd: Started Session 5 of user root.
Apr 3 13:36:06 zbxll01 systemd: Started Session 6 of user root.
Apr 3 13:36:06 zbxll01 systemd-logind: New session 6 of user root.
Apr 3 13:36:35 zbxll01 systemd: Started Session 7 of user root.
Apr 3 13:36:35 zbxll01 systemd-logind: New session 7 of user root.
Apr 3 13:37:48 zbxll01 systemd-logind: Removed session 7.

I do see both agent and server running, but for a brief period of time.

Thanks!

[tim.mooney]

Ok, the problem is the server is is crashing with a segmentation violation:

 10853:20200403:133239.564 Sending [system.users.num]
 10869:20200403:133239.564 End of zbx_es_init_env():SUCCEED
 10869:20200403:133239.564 In zbx_es_compile()
 10869:20200403:133239.564 End of zbx_es_compile():SUCCEED
 10869:20200403:133239.564 In zbx_es_execute()
 10869:20200403:133239.564 Got signal [signal:11(SIGSEGV),reason:1,refaddr:0x100b11873b6]. Crashing ...

You should open a ticket on the Zabbix bugtracker, and include your zabbix_server.log, as it has a stack trace in the log that might be useful to the developers.

[guille_pm]

Thanks a lot, I really appreciate it! I'll go ahead and do that.