ClickHouse setup

Zabbix can store history data in ClickHouse as an alternative to a relational database.

This guide covers the setup for the supported versions of ClickHouse. If you're using a different version, some functionality may not work as intended.

ClickHouse can store the following value types:

Item value type Database table ClickHouse type
Numeric (unsigned) history_uint uint
Numeric (float) history dbl
Character history_str str
Log history_log log
Text history_text text
Binary history_bin not supported by Zabbix
JSON history_json json

ClickHouse does not accept JSON arrays. A JSON value must be either a single object, or a set of objects. Additionally, ClickHouse handles JSON keys with NULL the same as missing keys.

Important notes

Configuring ClickHouse

You need to create and configure a Zabbix database and user, and import the database schema.

This guide gives instructions for Docker or package installations of ClickHouse.

Docker

1. Create and configure the Zabbix database and user when running the ClickHouse container:

sudo docker run -d \
  --name clickhouse \
  -e CLICKHOUSE_DB=zabbix \
  -e CLICKHOUSE_USER=zabbix \
  -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 \
  -e CLICKHOUSE_PASSWORD=<password> \
  -p 8123:8123/tcp \
  -p 9000:9000/tcp \
  --cap-add=SYS_NICE \
  --cap-add=NET_ADMIN \
  --cap-add=IPC_LOCK \
  --ulimit nofile=262144:262144 \
  -v clickhouse_data:/var/lib/clickhouse \
  clickhouse/clickhouse-server:26.4

2. Confirm that ClickHouse is working and you can connect to it:

sudo docker exec -it clickhouse \
  clickhouse-client \
  --query "SELECT version()"

# 26.4.4.38

3. Import the database schema using the history_all.sh script from your Zabbix directory:

./usr/share/zabbix/sql-scripts/clickhouse/history_all.sh \
  --user zabbix \
  --password <password> \
  --db zabbix \
  --server http://localhost:8123

The data storage period (Time-To-Live, or TTL) for ClickHouse is, by default, 31 days. To change it, use the --ttl option when importing the database schema (e.g., --ttl 604800 for 7 days), or configure it later.

4. Verify that tables were created:

sudo docker exec -it clickhouse \
  clickhouse-client \
  --user zabbix \
  --password <password> \
  --query "SHOW TABLES FROM zabbix"

# history
# history_json
# history_log
# history_str
# history_text
# history_uint

Packages

1. Start ClickHouse:

sudo service clickhouse-server start

2. Create and configure the Zabbix database and user:

clickhouse-client --user default

localhost :) CREATE DATABASE IF NOT EXISTS zabbix
localhost :) CREATE USER IF NOT EXISTS zabbix IDENTIFIED WITH sha256_password BY '<password>'
localhost :) GRANT CREATE, ALTER, DROP, INSERT, SELECT, UPDATE, OPTIMIZE ON zabbix.* TO zabbix
localhost :) quit

3. Import the database schema using the history_all.sh script from your Zabbix directory:

./usr/share/zabbix/sql-scripts/clickhouse/history_all.sh \
  --user zabbix \
  --password <password> \
  --db zabbix \
  --server http://localhost:8123

The data storage period (Time-To-Live, or TTL) for ClickHouse is, by default, 31 days. To change it, use the --ttl option when importing the database schema (e.g., --ttl 604800 for 7 days), or configure it later.

4. Verify that tables were created:

clickhouse-client \
  --user zabbix \
  --password <password> \
  --query "SHOW TABLES FROM zabbix"

# history
# history_json
# history_log
# history_str
# history_text
# history_uint

Configuring Zabbix server

In your Zabbix server configuration file (zabbix_server.conf), set the HistoryProvider parameter.

For example, to store all supported types of values in ClickHouse:

HistoryProvider=clickhouse;value_types="uint,dbl,str,log,text,json",url=http://localhost:8123,db=zabbix,username=zabbix,password=<password>

After making changes, restart Zabbix server:

systemctl restart zabbix-server

Configuring Zabbix frontend

In your Zabbix frontend configuration file (zabbix.conf.php), set the $HISTORY_PROVIDERS variable to match the server configuration:

$HISTORY_PROVIDERS[] = [
  'types' => ['uint','dbl','str','log','text','json'],
  'provider' => 'clickhouse',
  'url' => 'http://localhost:8123',
  'db' => 'zabbix',
  'username' => 'zabbix',
  'password' => '<password>'
];

Additional configuration

The steps below are optional. You do not need them for a basic setup.

Configuring ClickHouse data storage period

The data storage period (Time-To-Live, or TTL) for ClickHouse is, by default, 31 days. To change it, run the commands below.

The examples below use Docker. If you installed ClickHouse using packages, run the queries in the ClickHouse client directly.

1. Alter the table (replace history_json and 3600 with the values you need):

sudo docker exec -it clickhouse \
  clickhouse-client \
  --user zabbix \
  --password <password> \
  --query "ALTER TABLE zabbix.history_json MODIFY TTL clock_ns + toIntervalSecond(3600)"

2. Apply the changes immediately:

sudo docker exec -it clickhouse \
  clickhouse-client \
  --user zabbix \
  --password <password> \
  --query "OPTIMIZE TABLE zabbix.history_json MATERIALIZE TTL"

3. Verify that the data storage period changed:

sudo docker exec -it clickhouse \
  clickhouse-client \
  --user zabbix \
  --password <password> \
  --query "SHOW CREATE TABLE zabbix.history_json"

# CREATE TABLE zabbix.history_json
# (
#     `itemid` UInt64,
#     `clock_ns` DateTime64(9),
#     `value` JSON,
#     `value_str` String
# )
# ENGINE = MergeTree
# PARTITION BY toDate(clock_ns)
# PRIMARY KEY (itemid, clock_ns)
# ORDER BY (itemid, clock_ns)
# TTL clock_ns + toIntervalSecond(3600)
# SETTINGS index_granularity = 8192

4. Restart Zabbix server to refresh the data storage period in Administration > Housekeeping:

systemctl restart zabbix-server

Configuring ClickHouse clusters

This section provides configuration steps for ClickHouse clusters with replication and sharding.

The steps below use Docker and the ClickHouse cluster_2S_2R from ClickHouse documentation (running ClickHouse 26.4.4.38 with ClickHouse Keeper).

Before you start, it is recommended to put a load balancer in front of your ClickHouse cluster for Zabbix. Zabbix server and frontend configuration supports only a single ClickHouse URL, so it can only point to one node. If that node goes down, a load balancer will redirect Zabbix to a healthy node instead, through that same URL.

1. Create the Zabbix database:

sudo docker exec -it clickhouse-01 \
  clickhouse-client \
  --user default \
  --query "CREATE DATABASE IF NOT EXISTS zabbix ON CLUSTER cluster_2S_2R ENGINE = Replicated('/clickhouse/databases/zabbix', '{shard}', '{replica}')"

The ON CLUSTER clause reaches only nodes that are part of the cluster when you run the command. If you add nodes to your cluster later, run this command again.

2. Create and configure the Zabbix user:

sudo docker exec -it clickhouse-01 \
  clickhouse-client \
  --user default \
  --query "CREATE USER IF NOT EXISTS zabbix ON CLUSTER cluster_2S_2R IDENTIFIED WITH sha256_password BY '<password>'"

sudo docker exec -it clickhouse-01 \
  clickhouse-client \
  --user default \
  --query "GRANT ON CLUSTER cluster_2S_2R CREATE, ALTER, DROP, INSERT, SELECT, UPDATE, OPTIMIZE ON zabbix.* TO zabbix"

3. Import the database schema using the history_all.sh script from your Zabbix directory:

./usr/share/zabbix/sql-scripts/clickhouse/history_all.sh \
  --user zabbix \
  --password <password> \
  --db zabbix \
  --server http://localhost:8123 \
  --engine "ReplicatedMergeTree()"

The data storage period (Time-To-Live, or TTL) for ClickHouse is, by default, 31 days. To change it, use the --ttl option when importing the database schema (e.g., --ttl 604800 for 7 days), or configure it later.

If your ClickHouse cluster requires explicit replica paths (for example, if your ClickHouse server configuration does not set default paths for replicated tables), run the schema scripts (history_schema.sh, history_uint_schema.sh, etc.) individually and set a different path for each table:

./usr/share/zabbix/sql-scripts/clickhouse/history_schema.sh \
  --user zabbix \
  --password <password> \
  --db zabbix \
  --server http://localhost:8123 \
  --engine "ReplicatedMergeTree('/clickhouse/tables/{shard}/history', '{replica}')"

For more details on ClickHouse replication, see Replicating data and Replicated* table engines in ClickHouse documentation.

4. Configure sharding for each table type. Rename the tables and create a Distributed table with the original name, pointing to the renamed table. This Distributed table is what Zabbix actually reads from and writes to; the table splits data across shards automatically. For example, for the history table:

sudo docker exec -it clickhouse-01 \
  clickhouse-client \
  --user default \
  --query "RENAME TABLE zabbix.history TO zabbix.history_local"

sudo docker exec -it clickhouse-01 \
  clickhouse-client \
  --user default \
  --query "CREATE TABLE zabbix.history AS zabbix.history_local ENGINE = Distributed('cluster_2S_2R', 'zabbix', 'history_local', itemid)"

For more details on ClickHouse sharding, see Distributed table engine in ClickHouse documentation.

5. Configure Zabbix server.

6. Configure Zabbix frontend.

7. Verify that Zabbix data is spread across shards. The row count from zabbix.history_local may change between repeated checks, since Zabbix server continuously inserts new data. Run this command on each node:

sudo docker exec -it clickhouse-01 \
  clickhouse-client \
  --user default \
  --query "SELECT count(*) FROM zabbix.history_local"

# 2446 (clickhouse-01)
# 2471 (clickhouse-02)
# 2494 (clickhouse-03)

8. Check that replication is in sync (active_replicas should match total_replicas):

sudo docker exec -it clickhouse-01 \
  clickhouse-client \
  --user default \
  --query "SELECT database, table, is_leader, total_replicas, active_replicas FROM system.replicas WHERE database = 'zabbix'"

# zabbix  history_json_local  1 2 2
# zabbix  history_local       1 2 2
# zabbix  history_log_local   1 2 2
# zabbix  history_str_local   1 2 2
# zabbix  history_text_local  1 2 2
# zabbix  history_uint_local  1 2 2

Troubleshooting

The following steps may help you troubleshoot problems with your ClickHouse setup:

  1. Check ClickHouse or Zabbix server logs for errors.

  2. To identify slow queries, use the log_slow_queries option in the HistoryProvider Zabbix server configuration parameter.

  3. Verify that ClickHouse allows access from Zabbix server and Zabbix frontend.

  4. Query ClickHouse to see if the data collected by Zabbix is stored, for example:

sudo docker exec -it clickhouse \
  clickhouse-client \
  --user zabbix \
  --password <password> \
  --query "SELECT * FROM zabbix.history_uint WHERE itemid = 42269"