Zabbix preprocessing queue stuck and latest data not update

[baalchina]

Hi, all, I have a zabbix 6.0.6 system, update from 5.0 six month ago, and works fine. Last day I found zabbix server crashed, so I tried to change some settings.
Log like this:

 13629:20221206:143937.804 4: zabbix_server: preprocessing manager #1 [queued 458100, processed 14 values, idle 0.000000 sec during 5.003196 sec](MAIN_ZABBIX_ENTRY+0x943) [0x448f33]
 13629:20221206:143937.804 3: zabbix_server: preprocessing manager #1 [queued 458100, processed 14 values, idle 0.000000 sec during 5.003196 sec](daemon_start+0x1b2) [0x58fe92]
 13629:20221206:143937.804 2: zabbix_server: preprocessing manager #1 [queued 458100, processed 14 values, idle 0.000000 sec during 5.003196 sec](main+0x634) [0x4470d4]
 13629:20221206:143937.804 1: /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f08e74fd555]
 13629:20221206:143937.804 0: zabbix_server: preprocessing manager #1 [queued 458100, processed 14 values, idle 0.000000 sec during 5.003196 sec]() [0x4474f0]​

Then I tried some new parameter, after the preprocess queue to 100, and now the server do not crash again(300, 500 will crash). But the data do not update.

I found that in queue detain, the Zabbix preprocessing queue stuck for a very long time( more than 30 minutes), in latest data in zabbix server, the data is 99%, and the data is not updating too.

It looks like becaues the zabbix preporcessing queue is stuck, and all my hosts didnot update new data?

I think maybe this is because mysql problem, so after backup, I truncate all this tables including history, history_str,history_uint, trends, trends_uint and events. But still the same, the Zabbix preprocessing queue stuck after start zabbix server for a short time.

So, any suggestions for this problem? Thanks!

BTW, I had 1096 hosts, and 106K items. My server is CentOS 7, with 64G mem, 2 Xeon 2650 CPU, a 240G SSD. Before all this, eveythings works fine...

[Markku]

Does this match your case? https://support.zabbix.com/browse/ZBX-20590

Markku

[baalchina]

Thanks all, I have solve this problem and report back, After using `zabbix_server -R diaginfo` for sometimes, I found some top values of preprocessing diagnostic information, after check the items, all of them from some newly add elasticsearch hosts with official template. And when I clear the template from hosts, everything works fine.

BTW, I think this may still a bug, because I already had more than 10 server with official es template, and this stuck happend after three day after I had some new hosts.

Hope this will help.