Server Stops Updating

[chewie71]

I'm running Zabbix 1.8.4.

One thing I've noticed is that the server stops updating items...or processes them very slowly. Every day or two I have to stop/start the zabbix_server service to get it to start processing items normally again.

If I tail the zabbix_server.log, when it's running normally it processes several items a second, but when it's not working normally I may only see a few items go by per minute.

I've added a new alert to tell me when the Zabbix Queue has items over 1 hour old...I THINK that will tell me when the service has stopped processing. But I'd really like to figure out why it's doing it and fix it if possible.

Anyone have any recommendations?

Thanks,
Matt

[chewie71]

So for the last few nights it seems that it's getting hung up on the housekeeping maybe? It looks like I was getting data until about 1:15AM and then it stopped. Then mostly just housekeeping items appear in the log for the rest of the night.

  8216:20110415:011439.034 Item [zcs1:drive_backplane.temperature] error: Support of IPMI parameters was not compiled in
  8215:20110415:011440.076 Item [zcs1:fan_cooling[ft0.fm0.f0]] error: Support of IPMI parameters was not compiled in
  8217:20110415:011441.132 Item [zcs1:fan_cooling[ft0.fm0.f1]] error: Support of IPMI parameters was not compiled in
  8194:20110415:013020.049 Executing housekeeper
  8194:20110415:013627.729 Deleted 234005 records from history and trends
  8194:20110415:023727.734 Executing housekeeper
# SOME FAILING WEB SCENARIO STEPS IN HERE
  8194:20110415:035211.694 Executing housekeeper
  8194:20110415:041054.895 Deleted 196978 records from history and trends
# SOME FAILING WEB SCENARIO STEPS IN HERE
  8194:20110415:051154.900 Executing housekeeper
  8194:20110415:053604.284 Deleted 213176 records from history and trends
  8194:20110415:063704.289 Executing housekeeper
  8194:20110415:070728.270 Deleted 227853 records from history and trends
  8194:20110415:080828.275 Executing housekeeper

Except for maybe 20 lines of the failing web scenario steps I didn't include there.....that's all there is since 1:15AM.

I'm going to set the DEBUG level to 4 and see if I can get more info and figure out what's going on.

Matt

[chewie71]

I get LOTS of these messages in my log.

 28302:20110415:100649.225 History text buffer is full. Sleeping for 1 second.
 28310:20110415:100649.492 History text buffer is full. Sleeping for 1 second.
 28309:20110415:100649.492 History text buffer is full. Sleeping for 1 second.
 28313:20110415:100649.495 History text buffer is full. Sleeping for 1 second.
 28314:20110415:100649.495 History text buffer is full. Sleeping for 1 second.
 28308:20110415:100649.496 History text buffer is full. Sleeping for 1 second.
 28315:20110415:100649.497 History text buffer is full. Sleeping for 1 second.
 28317:20110415:100649.498 History text buffer is full. Sleeping for 1 second.
 28312:20110415:100649.498 History text buffer is full. Sleeping for 1 second.
 28316:20110415:100649.498 History text buffer is full. Sleeping for 1 second.
 28311:20110415:100649.575 History text buffer is full. Sleeping for 1 second.
 28298:20110415:100649.705 History text buffer is full. Sleeping for 1 second.
 28299:20110415:100650.040 History text buffer is full. Sleeping for 1 second.
 28302:20110415:100650.227 History text buffer is full. Sleeping for 1 second.

Could be I'm running into this issue?

https://support.zabbix.com/browse/ZBX-3603