Active Zabbix (no data) and Multiple Proxies for 'failover'

[iwidjaya]

I'm new with Zabbix and trying to understand how active zabbix agent and proxy works

I'm using active configuration with below configuration set:

ServerActive=proxy1,proxy2
Hostname=agenthostname
StartAgents=0

I apply a template to this agent but nothing is displayed in UI (no data) though I can see that it is actually executing the command (from zabbix agent log).

In addition to that there are many messages in agent log:

no active checks on server [proxy1], host [agenthostname] not found
no active checks on server [proxy2], host [agenthostname] not found

In proxy side, messages like below:
cannot send list of active checks to [IP]: host [agenthostname] not found

Then I turn off proxy1, and in the agent it show error:

active check configuration update from [proxy1:10051] started to fail (cannot connect to [[proxy1]:10051]: [111] Connection refused)

Looks like it was having communication though it was saying "no active checks"

Questions:
1. Is there any configuration missing from agent side?
2. The main purpose is to have redundant proxy so that data can be continuously available to zabbix server (High Availability). Is 'failover' concept here works to provide continuous data upstream to zabbix server?
3. If we need to monitor the zabbix agent availability, do we need to add "Server=" in agent configuration? If not then how we can monitor the agent ?

[zabx_ks]

Hi iwidjaya,

Try to set up config something like this:

Proxy1
zabbix_proxy.conf:
Server=Zabbix_server_IP
Hostname=proxy_hostname

zabbix_agentd.conf:
Server=Zabbix_server_IP,Proxy1_ip,Host1_ip
ServerActive=Proxy1_IP

Host1
zabbix_agentd.conf:
Server=Proxy1_IP

Frontend:
1) Administration - Proxies -> create proxy

2) Configuration - Hosts - Proxy1
set "Monitored by proxy" -> Proxy1

3) Configuration - Hosts - Host1
set "Monitored by proxy" -> Proxy1

zabx_ks

[iwidjaya]

Hi iwidjaya,

Try to set up config something like this:

Proxy1
zabbix_proxy.conf:
Server=Zabbix_server_IP
Hostname=proxy_hostname

zabbix_agentd.conf:
Server=Zabbix_server_IP,Proxy1_ip,Host1_ip
ServerActive=Proxy1_IP

Host1
zabbix_agentd.conf:
Server=Proxy1_IP

Frontend:
1) Administration - Proxies -> create proxy

2) Configuration - Hosts - Proxy1
set "Monitored by proxy" -> Proxy1

3) Configuration - Hosts - Host1
set "Monitored by proxy" -> Proxy1

zabx_ks

Just trying to understand here, why do we need to use IP instead of Hostname? And also why we add Host1_IP in the Server= field inside Proxy agent configuration?

[zabx_ks]

iwidjaya,

You can use whatever you want hostname or IP.
Wups, that Host1_IP shouldn’t be there, to much copy - paste

zabx_ks

[kloczek]

Sorry to say but you are trying to use zabbix not how it should be used but how you are thinking that it may be used.

If you have one host, two proxies and everything is connected to the same zabbix server.
In zabbix every host is distinguished by uni host name and physical path (connected straight to server or over proxy X1)
If you will have one active agent asking two proxies for monitoring cfg data and these requests will be connected to one server you will have kind of clash because host with the same hostname can be monitored only over one proxy. There is no automatic process of switching the same host to monitor hot suddenly over proxy X2. Even if you will do such change manually still on proxy X1 may beheld some not delivered monitoring data but on accepting those data from proxy X if host is in server configuration is using X2 proxy those data will be dropped because it will be not possible to find out to host with which one hostid these data need to be delivered.

What you are thing to archive is kind of HA on proxies layer. This is source of you wrong thinking about such HA.
All what you need to do to have such HA is to move service of the proxy with own VIP to another physical location (to new host). This how it is done HA from almost beginning and this hot it will be one probably in next ten years (as long as we still will be using the same protocols).

It s possible to use in agent set up multiple entries about servers/proxies but for the situation when two completely independent monitoring infrastructures would want to monitor the same agent. These proxies/servers may even deliver to the agent completely different set of metrics which needs to be monitored over such agent.

[iwidjaya]

Sorry to say but you are trying to use zabbix not how it should be used but how you are thinking that it may be used.

If you have one host, two proxies and everything is connected to the same zabbix server.
In zabbix every host is distinguished by uni host name and physical path (connected straight to server or over proxy X1)
If you will have one active agent asking two proxies for monitoring cfg data and these requests will be connected to one server you will have kind of clash because host with the same hostname can be monitored only over one proxy. There is no automatic process of switching the same host to monitor hot suddenly over proxy X2. Even if you will do such change manually still on proxy X1 may beheld some not delivered monitoring data but on accepting those data from proxy X if host is in server configuration is using X2 proxy those data will be dropped because it will be not possible to find out to host with which one hostid these data need to be delivered.

What you are thing to archive is kind of HA on proxies layer. This is source of you wrong thinking about such HA.
All what you need to do to have such HA is to move service of the proxy with own VIP to another physical location (to new host). This how it is done HA from almost beginning and this hot it will be one probably in next ten years (as long as we still will be using the same protocols).

It s possible to use in agent set up multiple entries about servers/proxies but for the situation when two completely independent monitoring infrastructures would want to monitor the same agent. These proxies/servers may even deliver to the agent completely different set of metrics which needs to be monitored over such agent.

Exactly, I'm looking for HA on proxies layer, which initially I thought that this can be achieved by adding 2nd proxy in ServerActive=. Looks like this is serving different purpose like what you have mentioned. I guess setting up RHEL clustered is the only option for achieving HA setup.

BTW, if we setup LB (Load Balancer) that hold 1 VIP of 2 (or more) Proxies, so from agent side it is pointing to VIP Proxy (rather than physical Proxy hostname), is this doable/workable? Has anyone tried this before and can share the outcome ?

[kloczek]

BTW, if we setup LB (Load Balancer) that hold 1 VIP of 2 (or more) Proxies, so from agent side it is pointing to VIP Proxy (rather than physical Proxy hostname), is this doable/workable? Has anyone tried this before and can share the outcome ?

If incoming traffic going from server to VIP with two passive proxies will be delivered to only one proxy as it is normally when you are using LB if proxy will be passive second one will be without configuration data. Now passive agents may theoretically work because only one proxy will be polling agents.
In case active agents probably half will be hitting proxy without cfg data.

In case of using active proxies as outgoing traffic will be NATed to VIP theoretically both proxies will receive exactly the same cfg data however any external checks or other except zabbix active agent items will be delivering to server two times more data and it will cause some unknown behaviour on the server.

As long as proxy is well defined object without any external dependencies duplicating this object will cause typical brain split state.

Theoretically with only using zabbix active items it may be working. However probably no one been trying to do this so you are entering on uncharted waters.
So forget in such case using ICMP. ODBC, JVM, IPMI and other metrics .. if it will be working at all.

Couple of months ago I had "strange issue" which caused that I've even opened case under paid zabbix support.
Suddenly we started receiving from one active proxy mixed signals that size of the queued items on the proxy is fluctuating between max (all items are queued) and minimum (only few with some known issues are queued), Other things as well started happening that ICMP checks started informing that time to time we have some packets loses.
After investigation with support over webex session they found that we had two active proxies with exactly the same settings but on two different addresses.
And it was true. Two days before on testing new proxy host procedure implemented on top of raw kickstart profile (only) I've redeployed physical box with exactly the same kickstart profiles as on production but with IP addresses assigned to non-prod network.
Before this I've blocked from this non-prod network all in/out traffic to agents but I forgot to block the same traffic to/from prod zabbix server.

Above case says for me only that even if it would be possible to have using LB and/without NAT active proxy and active agents still all internal proxies metrics will be more or less inconsistent.

So generally answer on your question is: no it will be not working.

Seems no one so far been even thinking about use zabbix infrastructure with HA organized that way and I'm really sure that number of issues which needs to be sorted out to have HA of the proxy secured that way is so big that probably even if you have paid support or if you will decide to sponsor develop new code to have such proxy HA zabbix developers will probably refuse to do this.
However this is only my opinion and you can always try to as them