Hi,
I have been experiencing a strange occurrence and no matter what I do, it happens. Every hour at 8 minutes after the hour, the db syncers and timer process rise to 100% and stay there for about 4 minutes. This used to cause the queue to rise to high amounts, but we migrated the database to it's own server with more I/O to resolve this issue. I have researched the forums and adjusted the trappers and pollers, etc to make suer they are not the issue. The issue is that when the db syncers increase to 100% and the database stops processing the queue. I can't figure out which actually happens first, if the database is causing the db syncers to go up or if the db synchers going up causing the database to hang.
The zabbix server and database server are separate large machines and not near capacity. We do process a large amount of data. I watch the database with innotop when this happens, and the qps just drop to around 300 for a few minutes. The bandwidth coming into the database also drops. I did try to debug zabbix, but did not find anything that stands out.
I am trying to found out what is running every hour that might be causing this issue. We have disabled all discoveries, housekeeper is disabled, there are no crons, etc.
I have been experiencing a strange occurrence and no matter what I do, it happens. Every hour at 8 minutes after the hour, the db syncers and timer process rise to 100% and stay there for about 4 minutes. This used to cause the queue to rise to high amounts, but we migrated the database to it's own server with more I/O to resolve this issue. I have researched the forums and adjusted the trappers and pollers, etc to make suer they are not the issue. The issue is that when the db syncers increase to 100% and the database stops processing the queue. I can't figure out which actually happens first, if the database is causing the db syncers to go up or if the db synchers going up causing the database to hang.
The zabbix server and database server are separate large machines and not near capacity. We do process a large amount of data. I watch the database with innotop when this happens, and the qps just drop to around 300 for a few minutes. The bandwidth coming into the database also drops. I did try to debug zabbix, but did not find anything that stands out.
I am trying to found out what is running every hour that might be causing this issue. We have disabled all discoveries, housekeeper is disabled, there are no crons, etc.
Attached Files