External Scripts & Proxies

[Clontarf[X]]

Hi All,

Wondering if anyone has any ideas on how to solve the following issue.

In a large environment (20k+ hosts), numerous proxies are required to handle the devices. Some of these devices have their data submitted by external scripts using zabbix_sender. With zabbix_sender you must specify the server to send to.

The problem; how can you determine which server to send to if you don't know ahead of time which proxy a device might be managed by? A proxy/server will reject values for devices which it doesn't monitor.

So, as you deploy more and more devices to more and more proxies, you end up in a situation where it's impossible to effectively use external scripts (or zabbix_sender) to send values for any given hosts.

Short of coding something into your script that queries the Zabbix API (or DB) to determine the correct "server" to send to, how do we solve this?

[Clontarf[X]]

FWIW: I raised a feature request for this. A method to either redirect zabbix_sender, or some kind of value forwarder when <host> is not monitored by <receiving_server|proxy>, but is known to be monitored by <server|proxy>.

https://support.zabbix.com/browse/ZBXNEXT-4276

[LenR]

When a host is assigned to a proxy, does it stay there forever or do they migrate? If the assignment is static, something like this might work; when the host arrives, use a API script to find the proxy, then use something like ansible to configure the zabbix_sender script on the new host. If you are using auto registration, this could be triggered by the action.

[Clontarf[X]]

In theory, the hosts shouldn't migrate, but we control this via our host management tool, so someone may decide to send x-thousand hosts to a different proxy at any time. API script sounds like it might be our only chance (we had considered this but wanted to see if there were other methods in use out there).

We've got numerous externalscripts that would need updating to use it, but I suppose once we've got something that's re-usable it shouldn't be too hard to implement. We're doing everything in ruby these days anyway so we could just write a module to do zabbix_send'ing with the smarts to determine the proxy.

Thanks for the brains

[LenR]

Maybe have all your scripts reference an environment variable for the proxy value, then all you have to do is get that set.

[Clontarf[X]]

Yeah we ended up doing the API lookup. Our concern initially was that it'd be too slow/intensive, but it turns out the lookup is actually very quick for the information we're after (hostid, proxy_hostid).

So, we've now got a tool that schedules an update of a CSV containing hosts and their proxy_hostid per server/proxy. We then wrote a ruby gem to use to call zabbix_sender and automatically determine the destinations via the CSV map. It does some smarts about creating zabbix cache files for each destination and then does what it needs to do.

Good discussion and ideas

[[email protected]]

I wrote the code that routes the zabbix sender data to the correct server or proxy for our environment.

I am also in the process or writing a snmptrap router that will route snmptrap data to the correct server or proxy.

I designed the above with performance in mind and utilise cache files that are generated on a schedule to determine the routes rather than hitting the API every time.

The above are only short term I think as Zabbix should / may get onto a log term internal routing process to cater for this as this will always be a large issue in environments like ours ie 10K hosts with multiple proxies.

With permission from work I may be able to share my design and code so message me if your interested.