2 Proxy load balancing

Overview

Proxy load balancing allows monitoring hosts by a proxy group with automated distribution of hosts between proxies and high proxy availability.

If one proxy from the proxy group goes offline, its hosts will be immediately distributed among other proxies having the least assigned hosts in the group.

Host redistribution happens only in "online" proxy groups. A proxy group is "online" if the configured minimum number of its proxies are online.

The proxy state is:

• online - if there was communication with it for the failover delay period (passive proxy responded to server requests and active proxy sent a request to server);
• offline - if there was no communication with it for the failover delay period;
• unknown - after proxy creation or server start.

Zabbix server also checks the balance between host-proxy assignments. If the balance is bad the group is marked for host reassignment, which will be done after 10xfailover delay period unless the balance is restored. The group is considered to be unbalanced if the number of hosts assigned to a proxy differs from the average (within the group) by at least 10 hosts and factor of 2.

Note that older version proxies and the hosts monitored by these proxies are excluded from re-balancing operations until they are upgraded.

Host reassignment

Zabbix server checks the balance between host assignments to the proxies. The group is considered "out of balance" if there is:

• host excess - a proxy has many more hosts than the group average;
• host deficit - a proxy has far fewer hosts than the group average.

The group is considered "out of balance" if the number of hosts assigned to the proxy is above/below the group average by more than 10 and a factor of 2. In this case the group is marked by the server for host reassignment after the grace period (10 x failover delay), if the balance is not restored.

The following table illustrates with example numbers when host reassignment is (or is not) triggered:

The proxy group manager will re-distribute hosts in proxy groups in the following way:

• calculate the average number of proxies per host;
• for proxies with host excess - move excess hosts to unassigned hosts;
• for proxies with host deficit - calculate the number of hosts needed to balance proxies;
• remove the missing number of hosts from proxies with most hosts;
• distribute unassigned hosts between proxies with least hosts.

Using Zabbix sender with proxy groups

Use the -g option when sending values from different hosts in the input file to a proxy in proxy group to avoid some of the values being sent to the wrong proxy.

Configuring proxy load balancing

To configure proxy load balancing for monitoring hosts:

1. Create a proxy group (see "Configuring a proxy group" below).

2. Make sure the proxy group is online.

3. Configure that hosts are monitored by proxy group (not individual proxies). You may use host mass update to move hosts from proxy to the proxy group.

4. Wait a few seconds for configuration update and for host distribution among proxies in the proxy group. Observe the change by refreshing the host list in Monitoring -> Hosts.

Note that adding all proxies of the group to the ServerActive agent parameter of monitored hosts is beneficial, but not mandatory. Proxy load balancing will work even with a single proxy added to ServerActive - active checks (and Zabbix sender data requests) will be redirected to the correct online proxy for the host based on the current proxy-host assignment.

Limitations

• SNMP traps are not supported by proxies in proxy group.
• Checks depending on external configuration must have the same configuration on all proxies in proxy group. That includes:
  • external checks - scripts;
  • database checks - odbc configuration.
• When monitored in proxy group the VMware hosts will be randomly spread between proxies in the group and will cause each proxy to cache all VMware data causing additional load to vCenter.

Possible firewall issues

Agents must always be allowed to reach all proxies at the firewall level. Consider the following scenarios:

• In Zabbix agent active checks, on agent startup, the first proxy responds and redirects to another proxy. The other proxy is not reachable because of a firewall problem and the communication stops in a state of waiting for the other proxy to respond. The root cause of this situation is that the first proxy knew that the other proxy was healthy for sure. This is not a problem if the first proxy fails and it will try different addresses configured in the "ServerActive" parameter.
• The HA setup has been stable for multiple months. Host rebalancing never happens; it is not needed. The agent does not need to validate the "backup" channel to any other proxies. In a failover scenario, it might fail because a firewall was modified half a year ago.

Configuring a proxy group

To configure a proxy group in Zabbix frontend:

• Go to: Administration → Proxy groups
• Click on Create proxy group