Making external checks run faster

[Linwood]

I've been experimenting with more external checks, and am curious whether there is better information on performance, notably...

Has anyone looked at language choice and impact, for example perl vs bash vs. php (or other)...? Does it seem to matter?

Related: it looked like, from what little I can tell in linux, that a check is nested about 5-6 layers in forks deep when it runs. Is that related to language? Is there a way to reduce that aspect?

One reason I have been using them is to do calculations. For example, I can calculate the snmp polling of CPUs on windows to an average and store one item, rather than 4 or 8, 12, 24 items which I average in a calculation. (This is a site that won't allow zabbix agent). But in a brief experiment converting to a perl external check, it looked like CPU overall went up polling that way, despite dramatically fewer items being managed (and maybe polled as it was one getbulk, whereas I've had to turn bulk off on windows in zabbix to keep it more reliable).

I see loadable modules as a possibility, but have not started down that road yet (I am building from source so it looks viable). But external checks are so easy... before I look to loadable modules, are there any good guidelines to make external check have less overhead?

[kloczek]

As long as one of the fundamental programming theorem says that "using even perfect programming language is possible to write bad code" it is not possible to answer on your so generalized questions.

Every modern OS uses more or less the same hardware on which every program executed and is not using network/storage IOs is affected mostly by one factor: CPU cache misses/hits ratio. More complicated code than higher probability that your code will be affected by exactly this factor.

[Linwood]

Perhaps I phrased the question badly...

I am not really asking which language (or any related aspect) runs best.

Let's assume that the time to execute the check itself, once running, is the same in all circumstances.

I am asking which presents the least overhead as invoked by zabbix on linux (if it matters ubuntu).

For example, I have no idea if firing up the perl interpreter incurs a lot more overhead than the bash interpreter.

Or perhaps more to the point, if internal to zabbix there is some way to do an external check that has persistence so each invocation does not have this overhead, some kind of service daemon for external checks. I assume this is really the loadable module.

[jan.garaj]

Use time command to measure CPU performance of your external check in the command line:

[root]# time cat test > /dev/null

real    0m0.006s
user    0m0.002s
sys     0m0.002s
[root]# time grep ".*" /test > /dev/null

real    0m0.017s
user    0m0.005s
sys     0m0.012s

Use statistic methods to eliminate any local anomalies and you will have reliable CPU performance results. Probably you should to measure also memory consumption to be fair.

Generally compiled languages (C, Go, ...) should be faster than interpreted languages (Perl, python, ...).

Generally loadable module should be the best option for Zabbix, but it still depends on the implementation.

Comparison (qps only) of module vs bash script https://github.com/monitoringartist/...rameter-script

[Linwood]

OK, I guess you indirectly answer what I am having trouble getting across -- there is nothing about how zabbix invokes these that changes substantially from just comparing them from the command prompt (in terms of performance).

For example (making this up and I am sure it is not true) if you said "zabbix has a compiled in python interpreter, so running python is going to be faster than any other interpreted language".

Or any variations on such a theme.

Thanks.