Zabbix High queue - hard to resolve

[bercerobry]

Hello Guys,

I am having trouble solving my Zabbix Server high queue issues. and I can't understand where the issue is.
All proxies have 5seconds and lesser check in times. Network connectivity is all good. I have attached several screenshots that my help in accelerating troubleshooting.
I also have checked on my host if there are some kind of throttling but they say that my VPS is performing optimally and they don't see any obvious issue.


VPS Specs (zabbix server) - Los Angeles
4 vcpu / 8GB ram / 200GB ssd

MariaDB: mysqld Ver 10.1.44-MariaDB-0ubuntu0.18.04.1 for debian-linux-gnu on x86_64 (Ubuntu 18.04)

Zabbix Version: Zabbix 4.4.10

I have 3 remote sites being monitored. Agents on those sites are monitored via the proxies and most monitors are snmpV2. Remote sites are in the Philippines.

• I can assure you that the zabbix performance graphs for all proxis are flat on the Queues.

Can you point me to places where I can check?

Zabbix Server Iostat this is while my queue are high.
#iostat -d -x -h 5





GRAPHS

[bercerobry]

Additional Info of my setup.

Even with Proxy of 5.7 vps, zabbix server still shows queue from this remote site which I don't understand why for such a very low amount of data.

[bercerobry]

Go to queue view https://www.zabbix.com/documentation...ig/items/queue
In up right corner select details and try to look at items that are in queue, find them in hosts (are they have some "info" messages?). Look in Zabbix, proxies and agent logs, try to find lines related to this items.

I couldn't find anything unusual, I am on default. On what debug level should I set the logs output?


--- Restarting the Zabbix server definitely zero's out the Queue but will be back when server is running over 24h. See below graph

[bercerobry]

I'm still having the same issues. the problem reappears after 3-4 days of Zabbix Server uptime

[bercerobry]

Queues grow because server cannot get data form monitored hosts.
In administration > queues what type of items have high values?
If you switch this view to "overview by proxy" - to what servers belongs high values? They are on one proxy, two proxy, server?

Hello Splitek, appreciate your input.

To answer, mostly are snmp stuffs. and even on a 7 host proxy I am still getting Queues from it which is unusual.
I have 3 remote sites, all of the proxies have varying queued items.

Also when I restart the whole zabbix server (including OS), queues are okay for 72 hours then it starts increasing again over time. I don't understand this behavior, because I am expecting high queue even after restarting the Virtual Server.

Restarting the MYSQL Daemon doesn't flatten out the queues, restarting the whole server does.

[bercerobry]

DB has nothing to do queues. In situations when something is wrong with DB you should see this in ie. "history syncer" and other parameters (have you other parameters ok?)
Do you telling that restarting Zabbix (not an OS) does not clear queues? really?

When server starts queues are empty. When some item are processed to long then next item wait (+1 item to queue), still processing this item? then again +1 to queue = +2 in sum, and so on... queue grow.

I quite do not understand your last sentence.

When I restart just the zabbix-server daemon, queues start at zero but within hours back to same problem.
When I restart the host OS, queues are okay for around 48 hours then on the 3rd day it starts to queue high again.