Zabbix Documentation 1.8

2.23.03.4In development:4.0Unsupported versions:1.82.02.43.2

User Tools

Site Tools


Sidebar


Ask questions to Zabbix Dev Team in person at the Zabbix Summit 2018!
manual:distributed_monitoring

15 Distributed Monitoring

Zabbix can be configured to support hierarchical distributed monitoring.

1 Goals

There are several goals of the distributed monitoring:

  • Get control of whole monitoring from a single or several locations

Zabbix administrator may control configuration of all Nodes from a single Zabbix WEB front-end.

  • Hierarchical monitoring

This is for monitoring of complex multi-level environments.

  • Monitor large complex environments

This is especially useful when monitoring several geographical locations.

  • Offload the overhead from busy Zabbix server

Monitoring thousands of hosts using single Zabbix server? This may be for you!

2009/08/13 17:06 · richlv

2 Overview

Zabbix provides effective and reliable way of monitoring distributed IT infrastructure. Configuration of the whole distributed setup can be done from a single location via common WEB interface.

Zabbix supports up-to 1000 (one thousand) Nodes in a distributed setup. Each Node is responsible for monitoring of its own Location. Node can be configured either locally or by its Master node which has a copy of configuration data of all Child Nodes. Configuration of Child Nodes can be done in off line mode, i.e. when there are no connectivity between Master and Child Node.

Hierarchical distributed monitoring allows having tree-like structure of Nodes. Each Node reports to its Master Node only.

All Nodes may work even in case of communication problems. Historical information and events are stored locally. When communication is back, Child Nodes will optionally send the data to Master Node.

New Nodes can be attached to and detached from the Zabbix distributed setup without any loss of functionality of the setup. No restart of any Node required.

Each Node has its own configuration and works as a normal Zabbix Server.

2009/08/13 17:07 · richlv

3 Configuration

3.1 Configuration of Nodes

Node configuration is performed in Administration → DM section.

Parameters of a Node:

ParameterDescription
Name Unique node name.
Id Unique Node ID.
Type Local – Local node
Remote – Remote node
Time zone Time zone of the Node. Zabbix automatically converts time stamps to local timezone when transferring time related data across nodes.
IP Node IP address. Zabbix trapper must be listening on this IP address.
Port Node Port number. Zabbix trapper must be listening on this port number. Default is 10051.
Do not keep history older than (in days) For non local historical data only. Zabbix won't keep history of the node longer than N days.
Do not keep trends older than (in days) For non local trend data only. Zabbix won't keep trends of the node longer than N days.

3.2 Simple configuration

Our simple configuration consists of a Central Node and a Child Node.

Central Node will have total control over configuration of Child Node. Child Node will report to central node events, history and trends.

Central Node will have NodeID=1, while Child Node's NodeID=2.

Central Node IP: 192.168.3.2
Child Node IP: 192.168.3.5

For Central Node

Step 1 Install Zabbix.

Follow standard installation instructions to create database, install Zabbix frontend and binaries.

Step 2 Setup NodeID in server configuration file.

In file zabbix_server.conf:

NodeID=1

Step 3 Convert database data.

Zabbix server has to be executed to convert unique IDs for use by first node.

cd bin
./zabbix_server -n 1 -c /etc/zabbix/zabbix_server.conf
Converting tables .................................................................. done.

Conversion completed.

This should be executed only once. This option is not required to start Zabbix server! Running Zabbix server with the -n option does not start the server process.

Step 4 Configure Node parameters.

Step 5 Add child node.

Step 6 Start Master Node.

We should see NodeID in startup messages of server log file:

31754:20070629:150342 server #16 started [Node watcher. Node ID:1]
For Child Node

Step 1 Install Zabbix.

Follow standard installation instructions to create database, install Zabbix frontend and binaries.

Step 2 Setup NodeID in server configuration file.

In file zabbix_server.conf:

NodeID=2

Step 3 Convert database data.

Zabbix server has to covert all IDs to unique ones for the second node.

cd bin
./zabbix_server -n 2 -c /etc/zabbix/zabbix_server.conf
Converting tables .................................................................. done.

Conversion completed.

This should be executed only once. This option is not required to start Zabbix server!

Step 4 Configure Node parameters.

Step 5 Add master node.

Step 6 Start Child Node.

We should see NodeID in startup messages of server log file:

27524:20070629:150622 server #9 started [Node watcher. Node ID:2]
Does it work?

Selection of active nodes will appear automatically after nodes are defined:

Add host for monitoring for Child Node node and see events coming to Master Node:

3.3 More complex setup

The setup consists of seven Nodes. Each Node may be configured either locally (using local WEB interface) or from one of its Master Nodes.

In this example, Riga (node 4) will collect events from all child nodes. It may also optionally collect historical information as well.

2009/08/13 17:08 · richlv

4 Platform independence

A node may use its own platform (OS, hardware) and database engine independently of other nodes. Also child nodes can be installed without Zabbix frontend.

It may be practical to use less powerful hardware with Zabbix server running SQLite or MySQL MyISAM while nodes of higher levels may use combination of a better hardware with MySQL InnoDB, Oracle or PostgreSQL backend.

2009/08/13 17:20 · richlv

5 Configuration of a single Node

Every Node in distributed environment must be properly configured to have a unique Node ID. Additional steps

Step 1

Follow standard installation procedure.

Follow standard installation procedure but do not start Zabbix Server. Zabbix front end must be installed and configured. Zabbix database must be created and populated with data from data.sql.

Step 2

Configure zabbix_server.conf.

Add NodeID to Zabbix Server configuration file. NodeID must be a unique Node ID.

Step 3

Configure Master and Child Nodes.

Use Zabbix Frontend to configure details of Nodes having direct communication with the Node. Make sure that all IP addresses and port numbers are correct.

Step 4

Start Zabbix Node.

Start Zabbix Server:

shell> ./zabbix_server

If everything was configured properly, Zabbix node will automatically start configuration and data exchange with all nodes in distributed setup. You may see the following messages in server log file:

...
11656:20061129:171614 NODE 2: Sending data of node 2 to node 1 datalen 3522738
11656:20061129:171614 NODE 2: Sending data of node 2 to node 1 datalen 20624
...
2009/08/13 17:22 · richlv

6 Switching between nodes

When connecting to a node in distributed setup, a list of available child nodes is accessible in right-upper corner of the GUI. It displays current node.

All information available in the GUI belongs to the selected node.

2009/08/13 17:23 · richlv

7 Data flow

7.1 Child to Master

Each Child Node periodically sends configuration changes, historical data and events to its Master Node.

DataFrequency
Configuration changesEvery 120 seconds.
EventsEvery 10 seconds.
HistoryEvery 10 seconds.

Child Node will resend data in case of communication problems.

Trends are calculated locally based on received historical data.

Zabbix does not send operational data across the nodes. For example, item-related information (last check, last value, etc) exists only locally.

Sending of Events and History can be controlled by configuration parameters NodeNoEvents and NodeNoHistory.

7.2 Master to Child

Each Master Node (a node with at least one child) periodically sends configuration changes to Child Nodes either directly or via other Child Nodes directly connected to the Master Node.

DataFrequency
Configuration changesEvery 120 seconds.

Zabbix does not send configuration of a Master Node to Childs.

7.3 Firewall settings

Inter-node communications use TCP protocol only.

Data flowSource portDestination port
Child to MasterAny10051

This is default port used by Zabbix trapper process.

2009/08/13 17:23 · richlv

8 Performance considerations

Any node requires more processing resources in a distributed setup. Master Node must be powerful enough to process and store not only local data but also data received from its all Child Nodes. Network communications must be also fast enough for timely transfer of new data.

2009/08/13 17:31 · richlv