Zabbix Server crashes

[bruno.pereira]

Hi community,

I've been having some issues with Zabbix Server stability. Here are the logs related to said issues.

#zabbix server logs
1156:20240904:052846.155 [Z3005] query failed: [2013] Lost connection to MySQL server during query [select ha_nodeid,name,status,lastaccess,address,port,ha_s essionid from ha_>
1156:20240904:052846.155 [Z3005] query failed: [2013] Lost connection to MySQL server during query [rollback;]
1216:20240904:052846.372 [Z3005] query failed: [1053] Server shutdown in progress [select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,a>
1216:20240904:052846.372 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
1216:20240904:052846.372 database is down: reconnecting in 10 seconds
1240:20240904:052846.533 [Z3005] query failed: [2013] Lost connection to MySQL server during query [select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.ret>
1240:20240904:052846.533 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
1240:20240904:052846.533 database is down: reconnecting in 10 seconds
1145:20240904:052846.573 Got signal [signal:15(SIGTERM),sender_pid:91689,sender_uid:0,r eason:0]. Exiting ...
1212:20240904:052846.574 [Z3005] query failed: [2013] Lost connection to MySQL server during query [insert into history (itemid,clock,ns,value) values (23266,1725427726,57325>
]
1212:20240904:052846.574 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
1212:20240904:052846.574 database is down: reconnecting in 10 seconds
1256:20240904:062846.574 thread stopped
1196:20240904:052846.574 [2] thread stopped [preprocessing worker #2]
1196:20240904:052846.574 [1] thread stopped [preprocessing worker #1]
1196:20240904:052846.575 [3] thread stopped [preprocessing worker #3]
1216:20240904:052846.575 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
1262:20240904:052846.575 thread stopped
1254:20240904:052846.575 thread stopped
1156:20240904:052846.575 HA manager has been paused
1156:20240904:052846.575 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
1216:20240904:052846.575 database is down: reconnecting in 10 seconds
1211:20240904:052846.575 thread stopped [discovery worker #4]
1211:20240904:052846.575 thread stopped [discovery worker #5]
1240:20240904:052846.575 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
1240:20240904:052846.575 database is down: reconnecting in 10 seconds
1211:20240904:052846.575 thread stopped [discovery worker #3]
1211:20240904:052846.575 thread stopped [discovery worker #2]
1211:20240904:052846.575 thread stopped [discovery worker #1]
1214:20240904:052847.574 syncing history data in progress...

#mysql logs
2024-09-04T05:28:45.852498Z 0 [System] [MY-013172] [Server] Received SHUTDOWN from user <via user signal>. Shutting down mysqld (Version: 8.0.39-0ubuntu0.24.04.1).
2024-09-04T05:28:47.860151Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 56 user: 'zbx_dev'.
2024-09-04T05:28:47.860264Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 272 user: 'zbx_dev'.
2024-09-04T05:28:47.860379Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 9 user: 'zbx_dev'.
2024-09-04T05:28:47.860974Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 273 user: 'zbx_dev'.
2024-09-04T05:28:47.861104Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 17 user: 'zbx_dev'.
2024-09-04T05:28:47.861273Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 51 user: 'zbx_dev'.
2024-09-04T05:28:47.861311Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 19 user: 'zbx_dev'.
2024-09-04T05:28:48.883963Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.39-0ubuntu0.24.04.1) (Ubuntu).

I've been searching online for answers with no luck so far, so I'm hoping that someone can help me.

Thanks in advance.

[bbrendon]

Check your sql process list and slow query logs.

[tim.mooney]

I've been having some issues with Zabbix Server stability. Here are the logs related to said issues.


1216:20240904:052846.372 [Z3005] query failed: [1053] Server shutdown in progress [select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,a>


#mysql logs
2024-09-04T05:28:45.852498Z 0 [System] [MY-013172] [Server] Received SHUTDOWN from user <via user signal>. Shutting down mysqld (Version: 8.0.39-0ubuntu0.24.04.1).

This isn't a Zabbix server stability issue... your MySQL server shut down. Zabbix server can't continue without a database.

You need to identify why your mysqld shut down. Resource issue? Auto-applied update at an inconvenient or unexpected time? Whatever it is, look first at your database.