Zabbix Queue backlog after radius failure

[ericg]

Zabbix Infrastructure
OS: Ubuntu 24.04
Zabbix: 7.4.10
Zabbix servers: 2 in HA
Zabbix Proxies: 5 in active mode, one is stand alone the other four are in proxy groups grouped in 2's
PostgreSQL: 18.3, w/timescaleDB 2.23.1
PGbouncer: 1.24.1
Compression: on after 7d

Issue experienced: Around 4am yesterday two of our Zabbix proxies stopped using their value cache and they started backlogging items. This all seems to coincide with our Radius server failing to authenticate requests during this time, the radius server has failed to auth at other times but not necessarily at this scale.
After Radius resumed working after a few moments, the two Zabbix proxies never caught up on the backlog of items you can seen in the queue of the screenshot I've included (both proxy 3 and 4 were about the same).
To resolve the backlog I Restarted the primary Zabbix server at 9:18 AM which failed over to the second Zabbix server, at this point certain metrics started recording again, but the queue stayed backlogged. At 1:12 PM I restarted both proxy 3 and 4 and that resolved their backlog of queued items.

Any suggestions for a bug this could be? or perhaps what I should see in the logs to help narrow down what the problem could be?

[ericg]

Any takers?

[mariusl]

Hi,

How is the communication between the proxies and the server set up? What encryption method is used? Are the proxies active or passive?
Depending on this, i would start by checking the relevant logs for any issues with communication between the components, for example due to encryption issues.

I have not used Radius with Zabbix, but i assume it's only used for authentication and authorization of user accounts? If so that shouldn't really have any impact on this unless you are running the processes with a custom user.