Delayed items

[TRNX]

Hello everybody,

I have problem with delayed items. I have trigger with .nodata function for one of items and trigger still switch ON/OFF. When I check last data time of this item, there is delay (+- 10 min).
I browsed for some solution so I edit some parameters in my.cnf file, but i don´t see any changes and I don´t know if my settings is good.

This is my statistics from Zabbix.

Hosts = 218	109 / 2 / 107
Items = 19041	15139 / 243 / 3659
Triggers = 10756	10661 / 95 [54 / 10607]

My server hardware is 2x CPU and 4GB RAM (and lot of disk space).

I have too many queued items. Is it normally? (Picture in attachment)


Thank everybody for every tip.

[Markku]

In the queues page, see the Details option in the dropdown menu on the upper right, maybe that gives you more insight what's happening.

Check that your agents (and server!) have time syncing configured correctly.

Markku

[TRNX]

Hello,

in details option I see only check which is delay and how long. I don´t know how can I diagnostic problem with this information.
Agents (hosts) and servers are the same time.

(I´m using active checks for every check.)

For me, it´s big problem because after night I have many e-mails (problems/resolved) with information that host didn´t recieve data.

[Markku]

You didn't mention the Zabbix server and agent versions, but take one problem host and update its Zabbix agent to the newest possible version in your Zabbix server release series. I remember having one single misbehaving active agent that had an older agent, and upgrading the agent solved the problems. I was left unclear if the previous installation was somehow faulty or if the agent version made the difference.

How many NVPS does your Zabbix server have?

Markku

[TRNX]

Execuse me, i forgot to mention about versions. Zabbix Server is 4.4.4. and Zabbix Agents on hosts are 4.4.1.
I updated one of hosts where I have problem with the delayed checks and now I´m waiting for result.

NVPS is now 18,85.

When you check my image with DB configuration - do you think everything is good?

Thank you very much.

[TRNX]

Update agent didn´t resolve my problem - still comes problem/resolved e-mails.

[Markku]

I don't see any specific problem in the DB configuration with that amount of NVPS, you could try limiting the buffer pool size to 2 GB since you are running the Zabbix server there as well. What does your Zabbix server performance metrics say? There are useful graphs in the Zabbix server template.

Anyway, when thinking about the increasing queues with active agents, it basically means that either the agents haven't sent the item data in time, or the Zabbix server cannot process the data. In my experience it is mostly the former (agent side problem). What kind of items are you having in the problem hosts? Aren't the specific items listed in the queue details page? I don't remember the exact output, can you copy something here?

Markku

[TRNX]

Hello,

I tried clean as much as possible items which is not supported. A lot of them was discovered network interfaces, so I created some exceptions.
I add some pictures - graphs and queued items. A lot of them are items of disk performance, but there's something out of it all.

Thank you

[TRNX]

I was limited for 5 files in attachments so there are next 5.

[TRNX]

Any ideas?

[Markku]

Have you already read https://www.zabbix.com/documentation...ig/items/queue?

What do you see in the zabbix_agentd.log (or zabbix_server.log)?

Markku

[TRNX]

Yes I read but there is not any tips or solution for this problem. It is only information page about Queue.
zabbix_server.log is full of informations about unsupported items - I think it´s ok.
In zabbix_agentd.log on Zabbix Server I don´t see any problems - only some records look like attack. But I don´t know.
zabbix_agentd.log on Windows Hosts looks ok, there are only records about unsupported items.

[Markku]

Ok so you have lots of failing/unsupported items configured, if I understand this correctly. How about just removing the conflicting items to see what effect they contribute to you problems?

Also do note that your Zabbix server's agent log indicated a couple of times that the agent could not connect to your server (127.0.0.1:10051), that's not a good sign at all. That kind of situation would definitely cause the queues to increase: agents cannot connect to the server.

Markku

[TRNX]

I am sorry for absence. I regulated some of unsupported items (I disabled them).
I don´t know why my agent could not connect to server. Server and agent is on the same host and server is still turn on. There is next screenshot from host (zabbix_agentd.log). There are some records with failed connect to server (localhost) and some connections from unknown public IP (attack?).

I still get many e-mails "Host did not recieve data" (.nodata trigger) and in queue tab is sometimes many delayed items. It is not items from one application but from some from every (network, disks, services, ...)

Thank you very much for your time and advice.

[Markku]

Based on the information you presented your Zabbix server is not able to receive the TCP connections from the agents.

Can you show the zabbix_server.log contents?

What special configurations have been made on the server, besides just installing the OS (which version?) and Zabbix components?

Are you the server administrator or does someone else run the server?

Markku