critical behavior with zabbix_agentd and custom parameters

[SAT QPass]

I have a huge list of custom parameters (over a 300). The problem seems to be that depsite the delay interval set for 60 seconds for most tests, zabbix_agentd is executing the list sequentially. All tests in this case are zabbix_active. No effort is being done to parrallelize the work even though there are 16 (the max number) of threads available.

Why is this?

In a separate question, can the maximum number of threads or child processes be increased beyond sixteen? StartAgents should not be hard capped as it is. Why is the hard limit set there? If you have a box that can handle the stress it should be a parameter that can be adjusted.

[SAT QPass]

Problem persists

I can definitively show by simply watching the process list that Zabbix and more importantly zabbix_agentd is executing the UserParameters in the zabbix_agentd.conf sequentially (1 at a time, despite the presence of 16 full threads) and completely ignoring any timing requests from the server.

Any suggestions? Thoughts, debug ideas? Method to provide more information that might be helpful?

[Alexei]

Actually this is not related to custom parameters, it is all about active checks.

ZABBIX agent has one dedicated process which executes all active checks (regardless of built-in or custom parameters) one by one. In case of large number of customer parameters the approach obviously becomes unefficient and creates a bottleneck which affects frequency of checks.

This problem should be addressed in next releases.

May I ask you where the 300 parameters comes from? Is it something that can be integrated into ZABBIX agent?

[SAT QPass]

Alexei,

Thanks for your explanation, that really helps explain what I was seeing.

I am almost sure it could be integrated, but it is not a trivial undertaking or at least I assume it is. This is related to our JMX polling methodology, every MBean is specific and each server has a lot of MBeans we are interested in, I assume if I were to switch these to all be regular zabbix_agent checks (not active) I would solve the problem short term or if I simply used zabbix_sender?

The best analogy I can offer is that JMX Mbeans are functionally equivalent to SNMP oids in terms of use and organization. You can make and poll hundreds, it is the polling that is expensive as it requires a definition for each MBean and some beans are "host" specific which makes it even tougher.

The only problem is we are now sliding out C into java and that just gets expensive as you are no doubt aware. I am sure there is a way to get these via C, I just have not found a way yet. I have not searched that hard.

Retooling to make them to be simple zabbix_agent checks (not active) should solve the problem short term.

Aaron