View Full Version : One child process died. Exiting
I am running the Zabbix server version 1:1.4.5-1. Sometimes it suddenly dies on me:
-----
669:20080528:145430 One child process died. Exiting ...
669:20080528:145432 ZABBIX Server stopped
-----
This is a VPS (Xen) only used by Zabbix. How can I further troubleshoot this? When I start the server again things work fine. I really don't want my server to go offline randomly.
nelsonab
29-05-2008, 09:36
I am running the Zabbix server version 1:1.4.5-1. Sometimes it suddenly dies on me:
-----
669:20080528:145430 One child process died. Exiting ...
669:20080528:145432 ZABBIX Server stopped
-----
This is a VPS (Xen) only used by Zabbix. How can I further troubleshoot this? When I start the server again things work fine. I really don't want my server to go offline randomly.
The best I can say is "Good luck!" Unless you can get it to happen on a regular basis there's pretty much no easy way to figure this out. It would be nice if the error said something like, "One poller child died. Exiting..." or something along those lines.
well, the only thing you can really do is use a higher debug level in the config file, so you can see what really happened (i suggest 3 if you do not have this already, 4 if you really want to debug, but dont forget to enlarge the LogFileSize parameter).
Most of the cases its database related ('database has gone away' or some query problem), but i haven't seen anyone on the forum speak of running zabbix under Xen so this could popup a problem in a completely different area.
It's happening fairly regularly now (once every two days or so). I increased the debug level and I actually installed monit to monitor the Zabbix-Server process and auto start it if it died ;)
Do you run a multi-node environment? If so, is all you data up-to-date on all servers?
I've encountered this a number of times, and usually find it's due to an item i've recently added, or somehting similar.
No I just have 1 central zabbix-server
jamied66
27-04-2009, 21:57
I have a box doing the same thing.
I'm getting this from my debug output.
32200:20090427:155438 In check_security()
32200:20090427:155438 Requested [system.cpu.util[,system,avg1]]
32197:20090427:155438 One child process died. Exiting ...
32197:20090427:155438 zbx_on_exit() called.
32199:20090427:155438 Got signal. Exiting ...
32203:20090427:155438 Got signal. Exiting ...
32202:20090427:155438 Got signal. Exiting ...
32201:20090427:155438 Got signal. Exiting ...
this is running agent 1.5.4 on an older Linux system (rhel 2.4.21-58.ELsmp)
any ideas are appreciated.
have you try upgrading to 1.6.4 ? as far as I know, 1.5.x versions are betas.
jamied66
27-04-2009, 22:24
thanks.
that fixed it.
the 1.6.4 agent works with no issues.
Vladimir
20-05-2009, 09:06
I have same trouble
13139:20090520:123932 server #12 started [Alerter]
13140:20090520:123932 server #13 started [Housekeeper]
13140:20090520:123932 Executing housekeeper
13141:20090520:123932 server #14 started [Timer]
/libexec/ld-elf.so.1: /usr/local/lib/libnetsnmp.so.16: Undefined symbol "dmalloc_strndup"
13143:20090520:123932 server #16 started [Node watcher. Node ID:0]
13144:20090520:123932 server #17 started [HTTP Poller]
/libexec/ld-elf.so.1: /usr/local/lib/libnetsnmp.so.16: Undefined symbol "dmalloc_strndup"
13146:20090520:123932 server #19 started [Escalator]
13127:20090520:123932 One child process died. Exiting ...
13127:20090520:123934 ZABBIX Server stopped. ZABBIX 1.6.4.
It happends after update net-snmp to version 5.4.2.1_5
P.S.
FreeBSD 7.1-RELEASE
Installed ports:
zabbix-1.6.4,1
net-snmp-5.4.2.1_5
php5-snmp-5.2.9
php5-5.2.9
mysql-server-5.0.77_1
P.P.S. Sorry for my bad English :rolleyes:
I am still periodically seeing this as well (v1.4) when the monitoring server is experiencing high load.
Vladimir
20-05-2009, 11:38
in my case zabbix simply not running.... :(
nelsonab
20-05-2009, 17:00
@vladimir What version are you running? I saw this recently happen consistantly with 1.6.4. The trapper processes seemed to be very unstable and as soon as a remote client would try and send something... boom! Zabbix died. The only way I could fix it for now was to reduce the number of trappers to 0, which fortunately was not a big deal in this environment as we don't use them.
@vladimir What version are you running? I saw this recently happen consistantly with 1.6.4. The trapper processes seemed to be very unstable and as soon as a remote client would try and send something... boom! Zabbix died. The only way I could fix it for now was to reduce the number of trappers to 0, which fortunately was not a big deal in this environment as we don't use them.
This is interesting. Could you provide us with more details about it. Zabbix server's log file would be excellent! ;)
Vladimir
20-05-2009, 17:41
nelsonab, yes, version is 1.6.4
but you wrong, zabbix died before it has been fully started
nelsonab
20-05-2009, 18:20
nelsonab, yes, version is 1.6.4
but you wrong, zabbix died before it has been fully started
Sometimes it would start, then die, sometimes run for a second and then die.
@Alexei I'll see what I can do. The environment is 1.6.4 server and 1.4.x clients.
Vladimir
21-05-2009, 06:05
Zabbix was started!!! :D
Port net-snmp was recompiled without "dmalloc debug memory allocator" and all become allright. :)
nelsonab
21-05-2009, 07:51
Nice, what is your environment aka what Linux distro and version?
Vladimir
21-05-2009, 07:59
nelsonab read my signature :rolleyes:
I am still experiencing this problem. Running latest Debian Lenny with Zabbix MySQL server 1.4.6-1. Any more suggestions how this can be fixed? A log with debug information does not show any extra clues. syslog however shows: Jun 3 13:47:49 monitor kernel: [1052093.583399] zabbix_server[1813]: segfault at 1 ip 08050e0b sp bfc452a0 error 4 in zabbix_server[8048000+4a000]