ClickHouse setup
Zabbix can store history data in ClickHouse as an alternative to a relational database.
This guide covers the setup for the supported versions of ClickHouse. If you're using a different version, some functionality may not work as intended.
ClickHouse can store the following value types:
| Item value type | Database table | ClickHouse type |
|---|---|---|
| Numeric (unsigned) | history_uint | uint |
| Numeric (float) | history | dbl |
| Character | history_str | str |
| Log | history_log | log |
| Text | history_text | text |
| Binary | history_bin | not supported by Zabbix |
| JSON | history_json | json |
ClickHouse does not accept JSON arrays. A JSON value must be either a single object, or a set of objects. Additionally, ClickHouse handles JSON keys with NULL the same as missing keys.
Important notes
- The housekeeper does not delete data from ClickHouse. To control how long data is kept, configure ClickHouse data storage period in ClickHouse.
- Zabbix does not calculate or store trends in ClickHouse. Consider extending the history storage period to preserve older data.
- If you want to migrate your history data from an existing Zabbix database (MySQL or PostgreSQL) to ClickHouse, see the ClickHouse schema and history migration scripts.
- ClickHouse is not supported for Zabbix proxies.
- If you have a ClickHouse cluster, see Configuring ClickHouse clusters.
Configuring ClickHouse
You need to create and configure a Zabbix database and user, and import the database schema.
This guide gives instructions for Docker or package installations of ClickHouse.
Docker
1. Create and configure the Zabbix database and user when running the ClickHouse container:
sudo docker run -d \
--name clickhouse \
-e CLICKHOUSE_DB=zabbix \
-e CLICKHOUSE_USER=zabbix \
-e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 \
-e CLICKHOUSE_PASSWORD=<password> \
-p 8123:8123/tcp \
-p 9000:9000/tcp \
--cap-add=SYS_NICE \
--cap-add=NET_ADMIN \
--cap-add=IPC_LOCK \
--ulimit nofile=262144:262144 \
-v clickhouse_data:/var/lib/clickhouse \
clickhouse/clickhouse-server:26.4
2. Confirm that ClickHouse is working and you can connect to it:
sudo docker exec -it clickhouse \
clickhouse-client \
--query "SELECT version()"
# 26.4.4.38
3. Import the database schema using the history_all.sh script from your Zabbix directory:
./usr/share/zabbix/sql-scripts/clickhouse/history_all.sh \
--user zabbix \
--password <password> \
--db zabbix \
--server http://localhost:8123
The data storage period (Time-To-Live, or TTL) for ClickHouse is, by default, 31 days.
To change it, use the --ttl option when importing the database schema (e.g., --ttl 604800 for 7 days), or configure it later.
4. Verify that tables were created:
sudo docker exec -it clickhouse \
clickhouse-client \
--user zabbix \
--password <password> \
--query "SHOW TABLES FROM zabbix"
# history
# history_json
# history_log
# history_str
# history_text
# history_uint
Packages
1. Start ClickHouse:
sudo service clickhouse-server start
2. Create and configure the Zabbix database and user:
clickhouse-client --user default
localhost :) CREATE DATABASE IF NOT EXISTS zabbix
localhost :) CREATE USER IF NOT EXISTS zabbix IDENTIFIED WITH sha256_password BY '<password>'
localhost :) GRANT CREATE, ALTER, DROP, INSERT, SELECT, UPDATE, OPTIMIZE ON zabbix.* TO zabbix
localhost :) quit
3. Import the database schema using the history_all.sh script from your Zabbix directory:
./usr/share/zabbix/sql-scripts/clickhouse/history_all.sh \
--user zabbix \
--password <password> \
--db zabbix \
--server http://localhost:8123
The data storage period (Time-To-Live, or TTL) for ClickHouse is, by default, 31 days.
To change it, use the --ttl option when importing the database schema (e.g., --ttl 604800 for 7 days), or configure it later.
4. Verify that tables were created:
clickhouse-client \
--user zabbix \
--password <password> \
--query "SHOW TABLES FROM zabbix"
# history
# history_json
# history_log
# history_str
# history_text
# history_uint
Configuring Zabbix server
In your Zabbix server configuration file (zabbix_server.conf), set the HistoryProvider parameter.
For example, to store all supported types of values in ClickHouse:
HistoryProvider=clickhouse;value_types="uint,dbl,str,log,text,json",url=http://localhost:8123,db=zabbix,username=zabbix,password=<password>
After making changes, restart Zabbix server:
systemctl restart zabbix-server
Configuring Zabbix frontend
In your Zabbix frontend configuration file (zabbix.conf.php), set the $HISTORY_PROVIDERS variable to match the server configuration:
$HISTORY_PROVIDERS[] = [
'types' => ['uint','dbl','str','log','text','json'],
'provider' => 'clickhouse',
'url' => 'http://localhost:8123',
'db' => 'zabbix',
'username' => 'zabbix',
'password' => '<password>'
];
Additional configuration
The steps below are optional. You do not need them for a basic setup.
Configuring ClickHouse data storage period
The data storage period (Time-To-Live, or TTL) for ClickHouse is, by default, 31 days. To change it, run the commands below.
The examples below use Docker. If you installed ClickHouse using packages, run the queries in the ClickHouse client directly.
1. Alter the table (replace history_json and 3600 with the values you need):
sudo docker exec -it clickhouse \
clickhouse-client \
--user zabbix \
--password <password> \
--query "ALTER TABLE zabbix.history_json MODIFY TTL clock_ns + toIntervalSecond(3600)"
2. Apply the changes immediately:
sudo docker exec -it clickhouse \
clickhouse-client \
--user zabbix \
--password <password> \
--query "OPTIMIZE TABLE zabbix.history_json MATERIALIZE TTL"
3. Verify that the data storage period changed:
sudo docker exec -it clickhouse \
clickhouse-client \
--user zabbix \
--password <password> \
--query "SHOW CREATE TABLE zabbix.history_json"
# CREATE TABLE zabbix.history_json
# (
# `itemid` UInt64,
# `clock_ns` DateTime64(9),
# `value` JSON,
# `value_str` String
# )
# ENGINE = MergeTree
# PARTITION BY toDate(clock_ns)
# PRIMARY KEY (itemid, clock_ns)
# ORDER BY (itemid, clock_ns)
# TTL clock_ns + toIntervalSecond(3600)
# SETTINGS index_granularity = 8192
4. Restart Zabbix server to refresh the data storage period in Administration > Housekeeping:
systemctl restart zabbix-server
Configuring ClickHouse clusters
This section provides configuration steps for ClickHouse clusters with replication and sharding.
The steps below use Docker and the ClickHouse cluster_2S_2R from ClickHouse documentation (running ClickHouse 26.4.4.38 with ClickHouse Keeper).
Before you start, it is recommended to put a load balancer in front of your ClickHouse cluster for Zabbix. Zabbix server and frontend configuration supports only a single ClickHouse URL, so it can only point to one node. If that node goes down, a load balancer will redirect Zabbix to a healthy node instead, through that same URL.
1. Create the Zabbix database:
sudo docker exec -it clickhouse-01 \
clickhouse-client \
--user default \
--query "CREATE DATABASE IF NOT EXISTS zabbix ON CLUSTER cluster_2S_2R ENGINE = Replicated('/clickhouse/databases/zabbix', '{shard}', '{replica}')"
The ON CLUSTER clause reaches only nodes that are part of the cluster when you run the command.
If you add nodes to your cluster later, run this command again.
2. Create and configure the Zabbix user:
sudo docker exec -it clickhouse-01 \
clickhouse-client \
--user default \
--query "CREATE USER IF NOT EXISTS zabbix ON CLUSTER cluster_2S_2R IDENTIFIED WITH sha256_password BY '<password>'"
sudo docker exec -it clickhouse-01 \
clickhouse-client \
--user default \
--query "GRANT ON CLUSTER cluster_2S_2R CREATE, ALTER, DROP, INSERT, SELECT, UPDATE, OPTIMIZE ON zabbix.* TO zabbix"
3. Import the database schema using the history_all.sh script from your Zabbix directory:
./usr/share/zabbix/sql-scripts/clickhouse/history_all.sh \
--user zabbix \
--password <password> \
--db zabbix \
--server http://localhost:8123 \
--engine "ReplicatedMergeTree()"
The data storage period (Time-To-Live, or TTL) for ClickHouse is, by default, 31 days.
To change it, use the --ttl option when importing the database schema (e.g., --ttl 604800 for 7 days), or configure it later.
If your ClickHouse cluster requires explicit replica paths (for example, if your ClickHouse server configuration does not set default paths for replicated tables), run the schema scripts (history_schema.sh, history_uint_schema.sh, etc.) individually and set a different path for each table:
./usr/share/zabbix/sql-scripts/clickhouse/history_schema.sh \
--user zabbix \
--password <password> \
--db zabbix \
--server http://localhost:8123 \
--engine "ReplicatedMergeTree('/clickhouse/tables/{shard}/history', '{replica}')"
For more details on ClickHouse replication, see Replicating data and Replicated* table engines in ClickHouse documentation.
4. Configure sharding for each table type.
Rename the tables and create a Distributed table with the original name, pointing to the renamed table.
This Distributed table is what Zabbix actually reads from and writes to; the table splits data across shards automatically.
For example, for the history table:
sudo docker exec -it clickhouse-01 \
clickhouse-client \
--user default \
--query "RENAME TABLE zabbix.history TO zabbix.history_local"
sudo docker exec -it clickhouse-01 \
clickhouse-client \
--user default \
--query "CREATE TABLE zabbix.history AS zabbix.history_local ENGINE = Distributed('cluster_2S_2R', 'zabbix', 'history_local', itemid)"
For more details on ClickHouse sharding, see Distributed table engine in ClickHouse documentation.
7. Verify that Zabbix data is spread across shards.
The row count from zabbix.history_local may change between repeated checks, since Zabbix server continuously inserts new data.
Run this command on each node:
sudo docker exec -it clickhouse-01 \
clickhouse-client \
--user default \
--query "SELECT count(*) FROM zabbix.history_local"
# 2446 (clickhouse-01)
# 2471 (clickhouse-02)
# 2494 (clickhouse-03)
8. Check that replication is in sync (active_replicas should match total_replicas):
sudo docker exec -it clickhouse-01 \
clickhouse-client \
--user default \
--query "SELECT database, table, is_leader, total_replicas, active_replicas FROM system.replicas WHERE database = 'zabbix'"
# zabbix history_json_local 1 2 2
# zabbix history_local 1 2 2
# zabbix history_log_local 1 2 2
# zabbix history_str_local 1 2 2
# zabbix history_text_local 1 2 2
# zabbix history_uint_local 1 2 2
Troubleshooting
The following steps may help you troubleshoot problems with your ClickHouse setup:
-
Check ClickHouse or Zabbix server logs for errors.
-
To identify slow queries, use the
log_slow_queriesoption in theHistoryProviderZabbix server configuration parameter. -
Verify that ClickHouse allows access from Zabbix server and Zabbix frontend.
-
Query ClickHouse to see if the data collected by Zabbix is stored, for example:
sudo docker exec -it clickhouse \
clickhouse-client \
--user zabbix \
--password <password> \
--query "SELECT * FROM zabbix.history_uint WHERE itemid = 42269"