Maximum size of distributed files queue to insert for trigger expression.

Maximum number of delayed inserts for trigger expression.

Filter of discoverable databases

Filter to exclude discovered databases

Filter of discoverable dictionaries

Filter to exclude discovered dictionaries

Maximum diff between log_pointer and log_max_index.

Maximum number of smth for trigger expression

Maximum number of parts per partition for trigger expression.

-

The port of ClickHouse HTTP endpoint

Maximum ClickHouse query time in seconds for trigger expression

Maximum size of the queue for operations waiting to be performed for trigger expression.

Replication lag across all tables for trigger expression.

Request scheme which may be http or https

-

Info about dictionaries

Filter:

- {#NAME} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DICT.MATCHES}

- {#NAME} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DICT.NOT_MATCHES}

Info about replicas

Filter:

- {#DB} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.MATCHES}

- {#DB} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.NOT_MATCHES}

Info about tables

Filter:

- {#DB} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.MATCHES}

- {#DB} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.NOT_MATCHES}

Get longest running query.

-

Preprocessing:

- DISCARD_UNCHANGED_HEARTBEAT: 10m

Preprocessing:

- REGEX: Ok\. 1

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- DISCARD_UNCHANGED_HEARTBEAT: 10m

Version of the server

Preprocessing:

- DISCARD_UNCHANGED_HEARTBEAT: 1d

Revision of the server.

Preprocessing:

- JSONPATH: $[?(@.metric == "Revision")].value.first()

Number of seconds since ClickHouse server start

Preprocessing:

- JSONPATH: $[?(@.metric == "Uptime")].value.first()

Number of queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.data.event == "Query")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of SELECT queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.event == "SelectQuery")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of INSERT queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertQuery")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Number of INSERT queries that are throttled due to high number of active data parts for partition in a MergeTree table."

Preprocessing:

- JSONPATH: $[?(@.metric == "DelayedInserts")].value.first()

Number of executing queries

Preprocessing:

- JSONPATH: $[?(@.metric == "Query")].value.first()

Number of executing background merges

Preprocessing:

- JSONPATH: $[?(@.metric == "Merge")].value.first()

The number of uncompressed bytes inserted in all tables.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Number of bytes (the number of bytes before decompression) read from compressed sources (files, network)."

Preprocessing:

- JSONPATH: $[?(@.event == "ReadCompressedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

The number of rows inserted in all tables.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertedRows")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Rows read for background merges.

Preprocessing:

- JSONPATH: $[?(@.event == "MergedRows")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Uncompressed bytes that were read for background merges

Preprocessing:

- JSONPATH: $[?(@.event == "MergedUncompressedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Clickhouse MergeTree table engine split each INSERT query to partitions (PARTITION BY expression) and add one or more PARTS per INSERT inside each partition,

after that background merge process run.

Preprocessing:

- JSONPATH: $[?(@.metric == "MaxPartCountForPartition")].value.first()

Number of connections to TCP server (clients with native interface).

Preprocessing:

- JSONPATH: $[?(@.metric == "TCPConnection")].value.first()

Number of connections to HTTP server.

Preprocessing:

- JSONPATH: $[?(@.metric == "HTTPConnection")].value.first()

Number of connections to remote servers sending data that was INSERTed into Distributed tables.

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedSend")].value.first()

Number of connections to MySQL server.

Preprocessing:

- JSONPATH: $[?(@.metric == "MySQLConnection")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

Number of connections from other replicas to fetch parts.

Preprocessing:

- JSONPATH: $[?(@.metric == "InterserverConnection")].value.first()

Network errors (timeouts and connection failures) during query execution, background pool tasks and DNS cache update.

Preprocessing:

- JSONPATH: $[?(@.event == "NetworkErrors")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of read (read, pread, io_getevents, etc.) syscalls in fly

Preprocessing:

- JSONPATH: $[?(@.metric == "Read")].value.first()

Number of write (write, pwrite, io_getevents, etc.) syscalls in fly

Preprocessing:

- JSONPATH: $[?(@.metric == "Write")].value.first()

"Total number of bytes allocated by the application."

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.allocated")].value.first()

Maximum number of bytes in physically resident data pages mapped by the allocator,

comprising all pages dedicated to allocator metadata, pages backing active allocations,

and unused dirty pages.

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.resident")].value.first()

"Total number of bytes in active extents mapped by the allocator."

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.mapped")].value.first()

"Total amount of memory (bytes) allocated in currently executing queries."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTracking")].value.first()

"Total amount of memory (bytes) allocated in background processing pool (that is dedicated for background merges, mutations and fetches).

Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa. This happens naturally due to caches for tables indexes and doesn't indicate memory leaks."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundProcessingPool")].value.first()

"Total amount of memory (bytes) allocated in background processing pool (that is dedicated for background moves). Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa.

This happens naturally due to caches for tables indexes and doesn't indicate memory leaks."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundMoveProcessingPool")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

"Total amount of memory (bytes) allocated in background schedule pool (that is dedicated for bookkeeping tasks of Replicated tables)."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundSchedulePool")].value.first()

Total amount of memory (bytes) allocated for background merges. Included in MemoryTrackingInBackgroundProcessingPool. Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa.

This happens naturally due to caches for tables indexes and doesn't indicate memory leaks.

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingForMerges")].value.first()

Number of pending files to process for asynchronous insertion into Distributed tables. Number of files for every shard is summed.

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedFilesToInsert")].value.first()

Connection retries in replicated DB connection pool

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedConnectionFailTry")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Connection failures after all retries in replicated DB connection pool"

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedConnectionFailAtAll")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Maximum replica queue delay relative to current time

Preprocessing:

- JSONPATH: $[?(@.metric == "ReplicasMaxAbsoluteDelay")].value.first()

Preprocessing:

- JSONPATH: $[?(@.metric == "ReplicasSumQueueSize")].value.first()

Number of Replicated tables that are currently in readonly state

due to re-initialization after ZooKeeper session loss

or due to startup without ZooKeeper configured.

Preprocessing:

- JSONPATH: $[?(@.metric == "ReadonlyReplica")].value.first()

Table size in bytes. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].bytes.first()

Number of parts of the table. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].parts.first()

Number of rows in the table. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].rows.first()

Database size in bytes.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}")].bytes.sum()

Whether the replica is in read-only mode.

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when re-initializing sessions in ZooKeeper, and during session re-initialization in ZooKeeper.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].is_readonly.first()

True if the ZooKeeper session expired

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].is_session_expired.first()

Number of data parts that will appear as the result of INSERTs or merges that haven't been done yet.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].future_parts.first()

Number of data parts in the queue for verification. A part is put in the verification queue if there is suspicion that it might be damaged.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].parts_to_check.first()

Size of the queue for operations waiting to be performed.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].queue_size.first()

Number of inserts of blocks of data that need to be made.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].inserts_in_queue.first()

Number of merges waiting to be made.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].merges_in_queue.first()

Maximum entry number in the log of general activity. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].log_max_index.first()

Maximum entry number in the log of general activity that the replica copied to its execution queue, plus one. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].log_pointer.first()

Total number of known replicas of this table. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].total_replicas.first()

Number of replicas of this table that have a session in ZooKeeper (i.e., the number of functioning replicas). (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].active_replicas.first()

Difference between log_max_index and log_pointer

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].replica_lag.first()

The amount of RAM the dictionary uses.

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].bytes_allocated.first()

Number of items stored in the dictionary.

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].element_count.first()

The percentage filled in the dictionary (for a hashed dictionary, the percentage filled in the hash table).

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].bytes_allocated.first()

- MULTIPLIER: 100

Number of sessions (connections) to ZooKeeper. Should be no more than one.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperSession")].value.first()

Number of watches (e.g., event subscriptions) in ZooKeeper.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperWatch")].value.first()

Number of requests to ZooKeeper in progress.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperRequest")].value.first()

Time spent in waiting for ZooKeeper operations.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperWaitMicroseconds")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- MULTIPLIER: 0.000001

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions that does not belong to user/hardware exceptions.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperOtherExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions caused by session moved/expired, connection loss, marshalling error, operation timed out and invalid zhandle state.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperHardwareExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions caused by no znodes, bad version, node exists, node empty and no children for ephemeral.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperUserExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Get information about the number of events that have occurred in the system.

Preprocessing:

- JSONPATH: $.data

Get metrics which can be calculated instantly, or have a current value format JSONEachRow

Preprocessing:

- JSONPATH: $.data

Get metrics that are calculated periodically in the background

Preprocessing:

- JSONPATH: $.data

Get information about settings that are currently in use.

Preprocessing:

- JSONPATH: $.data

- DISCARD_UNCHANGED_HEARTBEAT: 1h

-

Preprocessing:

- JSONPATH: $.data

-

Preprocessing:

- JSONPATH: $.data

-

Preprocessing:

- JSONPATH: $.data

-

Manual close: YES

-

Manual close: YES

-

Manual close: YES

Depends on:

- ClickHouse: Port {$CLICKHOUSE.PORT} is unavailable

ClickHouse version has changed. Ack to close.

Manual close: YES

Uptime is less than 10 minutes.

Manual close: YES

Zabbix has not received data for items for the last 30 minutes

Manual close: YES

Depends on:

- ClickHouse: Service is down

Clickhouse have INSERT queries that are throttled due to high number of active data parts for partition in a MergeTree, please decrease INSERT frequency

Manual close: YES

Descease INSERT queries frequency.

Clickhouse MergeTree table engine split each INSERT query to partitions (PARTITION BY expression)

and add one or more PARTS per INSERT inside each partition,

after that background merge process run, and when you have too much unmerged parts inside partition,

SELECT queries performance can significate degrade, so clickhouse try delay insert, or abort it.

Manual close: YES

Number of errors (timeouts and connection failures) during query execution, background pool tasks and DNS cache update is too high.

"Clickhouse servers and in config.xml

https://clickhouse.tech/docs/en/operations/table_engines/distributed/"

Manual close: YES

When replica have too much lag, it can be skipped from Distributed SELECT Queries without errors

and you will have wrong query results.

Manual close: YES

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when re-initializing sessions in ZooKeeper, and during session re-initialization in ZooKeeper.

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when re-initializing sessions in ZooKeeper, and during session re-initialization in ZooKeeper.

-

-

-

Number of sessions (connections) to ZooKeeper.

Should be no more than one, because using more than one connection to ZooKeeper may lead to bugs due to lack of linearizability (stale reads) that ZooKeeper consistency model allows.

ClickHouse configuration has been changed. Ack to close.

Manual close: YES

Maximum size of distributed files queue to insert for trigger expression.

Maximum number of delayed inserts for trigger expression.

Filter of discoverable databases

Filter to exclude discovered databases

Filter of discoverable dictionaries

Filter to exclude discovered dictionaries

Maximum diff between log_pointer and log_max_index.

Maximum number of smth for trigger expression

Maximum number of parts per partition for trigger expression.

-

The port of ClickHouse HTTP endpoint

Maximum ClickHouse query time in seconds for trigger expression

Maximum size of the queue for operations waiting to be performed for trigger expression.

Replication lag across all tables for trigger expression.

Request scheme which may be http or https

-

Info about dictionaries

Filter:

- {#NAME} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DICT.MATCHES}

- {#NAME} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DICT.NOT_MATCHES}

Info about replicas

Filter:

- {#DB} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.MATCHES}

- {#DB} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.NOT_MATCHES}

Info about tables

Filter:

- {#DB} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.MATCHES}

- {#DB} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.NOT_MATCHES}

Get longest running query.

-

Preprocessing:

- DISCARD_UNCHANGED_HEARTBEAT: 10m

Preprocessing:

- REGEX: Ok\. 1

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- DISCARD_UNCHANGED_HEARTBEAT: 10m

Version of the server

Preprocessing:

- DISCARD_UNCHANGED_HEARTBEAT: 1d

Revision of the server.

Preprocessing:

- JSONPATH: $[?(@.metric == "Revision")].value.first()

Number of seconds since ClickHouse server start

Preprocessing:

- JSONPATH: $[?(@.metric == "Uptime")].value.first()

Number of queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.data.event == "Query")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of SELECT queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.event == "SelectQuery")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of INSERT queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertQuery")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Number of INSERT queries that are throttled due to high number of active data parts for partition in a MergeTree table."

Preprocessing:

- JSONPATH: $[?(@.metric == "DelayedInserts")].value.first()

Number of executing queries

Preprocessing:

- JSONPATH: $[?(@.metric == "Query")].value.first()

Number of executing background merges

Preprocessing:

- JSONPATH: $[?(@.metric == "Merge")].value.first()

The number of uncompressed bytes inserted in all tables.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Number of bytes (the number of bytes before decompression) read from compressed sources (files, network)."

Preprocessing:

- JSONPATH: $[?(@.event == "ReadCompressedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

The number of rows inserted in all tables.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertedRows")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Rows read for background merges.

Preprocessing:

- JSONPATH: $[?(@.event == "MergedRows")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Uncompressed bytes that were read for background merges

Preprocessing:

- JSONPATH: $[?(@.event == "MergedUncompressedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Clickhouse MergeTree table engine split each INSERT query to partitions (PARTITION BY expression) and add one or more PARTS per INSERT inside each partition,

after that background merge process run.

Preprocessing:

- JSONPATH: $[?(@.metric == "MaxPartCountForPartition")].value.first()

Number of connections to TCP server (clients with native interface).

Preprocessing:

- JSONPATH: $[?(@.metric == "TCPConnection")].value.first()

Number of connections to HTTP server.

Preprocessing:

- JSONPATH: $[?(@.metric == "HTTPConnection")].value.first()

Number of connections to remote servers sending data that was INSERTed into Distributed tables.

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedSend")].value.first()

Number of connections to MySQL server.

Preprocessing:

- JSONPATH: $[?(@.metric == "MySQLConnection")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

Number of connections from other replicas to fetch parts.

Preprocessing:

- JSONPATH: $[?(@.metric == "InterserverConnection")].value.first()

Network errors (timeouts and connection failures) during query execution, background pool tasks and DNS cache update.

Preprocessing:

- JSONPATH: $[?(@.event == "NetworkErrors")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of read (read, pread, io_getevents, etc.) syscalls in fly

Preprocessing:

- JSONPATH: $[?(@.metric == "Read")].value.first()

Number of write (write, pwrite, io_getevents, etc.) syscalls in fly

Preprocessing:

- JSONPATH: $[?(@.metric == "Write")].value.first()

"Total number of bytes allocated by the application."

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.allocated")].value.first()

Maximum number of bytes in physically resident data pages mapped by the allocator,

comprising all pages dedicated to allocator metadata, pages backing active allocations,

and unused dirty pages.

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.resident")].value.first()

"Total number of bytes in active extents mapped by the allocator."

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.mapped")].value.first()

"Total amount of memory (bytes) allocated in currently executing queries."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTracking")].value.first()

"Total amount of memory (bytes) allocated in background processing pool (that is dedicated for background merges, mutations and fetches).

Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa. This happens naturally due to caches for tables indexes and doesn't indicate memory leaks."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundProcessingPool")].value.first()

"Total amount of memory (bytes) allocated in background processing pool (that is dedicated for background moves). Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa.

This happens naturally due to caches for tables indexes and doesn't indicate memory leaks."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundMoveProcessingPool")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

"Total amount of memory (bytes) allocated in background schedule pool (that is dedicated for bookkeeping tasks of Replicated tables)."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundSchedulePool")].value.first()

Total amount of memory (bytes) allocated for background merges. Included in MemoryTrackingInBackgroundProcessingPool. Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa.

This happens naturally due to caches for tables indexes and doesn't indicate memory leaks.

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingForMerges")].value.first()

Number of pending files to process for asynchronous insertion into Distributed tables. Number of files for every shard is summed.

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedFilesToInsert")].value.first()

Connection retries in replicated DB connection pool

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedConnectionFailTry")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Connection failures after all retries in replicated DB connection pool"

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedConnectionFailAtAll")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Maximum replica queue delay relative to current time

Preprocessing:

- JSONPATH: $[?(@.metric == "ReplicasMaxAbsoluteDelay")].value.first()

Preprocessing:

- JSONPATH: $[?(@.metric == "ReplicasSumQueueSize")].value.first()

Number of Replicated tables that are currently in readonly state

due to re-initialization after ZooKeeper session loss

or due to startup without ZooKeeper configured.

Preprocessing:

- JSONPATH: $[?(@.metric == "ReadonlyReplica")].value.first()

Table size in bytes. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].bytes.first()

Number of parts of the table. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].parts.first()

Number of rows in the table. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].rows.first()

Database size in bytes.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}")].bytes.sum()

Whether the replica is in read-only mode.

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when re-initializing sessions in ZooKeeper, and during session re-initialization in ZooKeeper.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].is_readonly.first()

True if the ZooKeeper session expired

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].is_session_expired.first()

Number of data parts that will appear as the result of INSERTs or merges that haven't been done yet.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].future_parts.first()

Number of data parts in the queue for verification. A part is put in the verification queue if there is suspicion that it might be damaged.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].parts_to_check.first()

Size of the queue for operations waiting to be performed.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].queue_size.first()

Number of inserts of blocks of data that need to be made.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].inserts_in_queue.first()

Number of merges waiting to be made.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].merges_in_queue.first()

Maximum entry number in the log of general activity. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].log_max_index.first()

Maximum entry number in the log of general activity that the replica copied to its execution queue, plus one. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].log_pointer.first()

Total number of known replicas of this table. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].total_replicas.first()

Number of replicas of this table that have a session in ZooKeeper (i.e., the number of functioning replicas). (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].active_replicas.first()

Difference between log_max_index and log_pointer

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].replica_lag.first()

The amount of RAM the dictionary uses.

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].bytes_allocated.first()

Number of items stored in the dictionary.

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].element_count.first()

The percentage filled in the dictionary (for a hashed dictionary, the percentage filled in the hash table).

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].bytes_allocated.first()

- MULTIPLIER: 100

Number of sessions (connections) to ZooKeeper. Should be no more than one.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperSession")].value.first()

Number of watches (e.g., event subscriptions) in ZooKeeper.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperWatch")].value.first()

Number of requests to ZooKeeper in progress.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperRequest")].value.first()

Time spent in waiting for ZooKeeper operations.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperWaitMicroseconds")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- MULTIPLIER: 0.000001

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions that does not belong to user/hardware exceptions.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperOtherExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions caused by session moved/expired, connection loss, marshalling error, operation timed out and invalid zhandle state.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperHardwareExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions caused by no znodes, bad version, node exists, node empty and no children for ephemeral.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperUserExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Get information about the number of events that have occurred in the system.

Preprocessing:

- JSONPATH: $.data

Get metrics which can be calculated instantly, or have a current value format JSONEachRow

Preprocessing:

- JSONPATH: $.data

Get metrics that are calculated periodically in the background

Preprocessing:

- JSONPATH: $.data

Get information about settings that are currently in use.

Preprocessing:

- JSONPATH: $.data

- DISCARD_UNCHANGED_HEARTBEAT: 1h

-

Preprocessing:

- JSONPATH: $.data

-

Preprocessing:

- JSONPATH: $.data

-

Preprocessing:

- JSONPATH: $.data

-

Manual close: YES

-

Manual close: YES

-

Manual close: YES

Depends on:

- ClickHouse: Port {$CLICKHOUSE.PORT} is unavailable

ClickHouse version has changed. Ack to close.

Manual close: YES

Uptime is less than 10 minutes.

Manual close: YES

Zabbix has not received data for items for the last 30 minutes

Manual close: YES

Depends on:

- ClickHouse: Service is down

Clickhouse have INSERT queries that are throttled due to high number of active data parts for partition in a MergeTree, please decrease INSERT frequency

Manual close: YES

Descease INSERT queries frequency.

Clickhouse MergeTree table engine split each INSERT query to partitions (PARTITION BY expression)

and add one or more PARTS per INSERT inside each partition,

after that background merge process run, and when you have too much unmerged parts inside partition,

SELECT queries performance can significate degrade, so clickhouse try delay insert, or abort it.

Manual close: YES

Number of errors (timeouts and connection failures) during query execution, background pool tasks and DNS cache update is too high.

"Clickhouse servers and in config.xml

https://clickhouse.tech/docs/en/operations/table_engines/distributed/"

Manual close: YES

When replica have too much lag, it can be skipped from Distributed SELECT Queries without errors

and you will have wrong query results.

Manual close: YES

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when re-initializing sessions in ZooKeeper, and during session re-initialization in ZooKeeper.

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when re-initializing sessions in ZooKeeper, and during session re-initialization in ZooKeeper.

-

-

-

Number of sessions (connections) to ZooKeeper.

Should be no more than one, because using more than one connection to ZooKeeper may lead to bugs due to lack of linearizability (stale reads) that ZooKeeper consistency model allows.

ClickHouse configuration has been changed. Ack to close.

Manual close: YES

Maximum size of distributed files queue to insert for trigger expression.

Maximum number of delayed inserts for trigger expression.

Filter of discoverable databases

Filter to exclude discovered databases

Filter of discoverable dictionaries

Filter to exclude discovered dictionaries

Maximum diff between log_pointer and log_max_index.

Maximum number of smth for trigger expression

Maximum number of parts per partition for trigger expression.

-

The port of ClickHouse HTTP endpoint

Maximum ClickHouse query time in seconds for trigger expression

Maximum size of the queue for operations waiting to be performed for trigger expression.

Replication lag across all tables for trigger expression.

Request scheme which may be http or https

-

Info about tables

Filter:

- {#DB} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.MATCHES}

- {#DB} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.NOT_MATCHES}

Info about replicas

Filter:

- {#DB} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.MATCHES}

- {#DB} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.NOT_MATCHES}

Info about dictionaries

Filter:

- {#NAME} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DICT.MATCHES}

- {#NAME} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DICT.NOT_MATCHES}

Get longest running query.

-

Preprocessing:

- DISCARD_UNCHANGED_HEARTBEAT: 10m

Preprocessing:

- REGEX: Ok\. 1

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- DISCARD_UNCHANGED_HEARTBEAT: 10m

Version of the server

Preprocessing:

- DISCARD_UNCHANGED_HEARTBEAT: 1d

Revision of the server.

Preprocessing:

- JSONPATH: $[?(@.metric == "Revision")].value.first()

Number of seconds since ClickHouse server start

Preprocessing:

- JSONPATH: $[?(@.metric == "Uptime")].value.first()

Number of queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.data.event == "Query")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of SELECT queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.event == "SelectQuery")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of INSERT queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertQuery")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Number of INSERT queries that are throttled due to high number of active data parts for partition in a MergeTree table."

Preprocessing:

- JSONPATH: $[?(@.metric == "DelayedInserts")].value.first()

Number of executing queries

Preprocessing:

- JSONPATH: $[?(@.metric == "Query")].value.first()

Number of executing background merges

Preprocessing:

- JSONPATH: $[?(@.metric == "Merge")].value.first()

The number of uncompressed bytes inserted in all tables.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Number of bytes (the number of bytes before decompression) read from compressed sources (files, network)."

Preprocessing:

- JSONPATH: $[?(@.event == "ReadCompressedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

The number of rows inserted in all tables.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertedRows")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Rows read for background merges.

Preprocessing:

- JSONPATH: $[?(@.event == "MergedRows")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Uncompressed bytes that were read for background merges

Preprocessing:

- JSONPATH: $[?(@.event == "MergedUncompressedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Clickhouse MergeTree table engine split each INSERT query to partitions (PARTITION BY expression) and add one or more PARTS per INSERT inside each partition,

after that background merge process run.

Preprocessing:

- JSONPATH: $[?(@.metric == "MaxPartCountForPartition")].value.first()

Number of connections to TCP server (clients with native interface).

Preprocessing:

- JSONPATH: $[?(@.metric == "TCPConnection")].value.first()

Number of connections to HTTP server.

Preprocessing:

- JSONPATH: $[?(@.metric == "HTTPConnection")].value.first()

Number of connections to remote servers sending data that was INSERTed into Distributed tables.

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedSend")].value.first()

Number of connections to MySQL server.

Preprocessing:

- JSONPATH: $[?(@.metric == "MySQLConnection")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

Number of connections from other replicas to fetch parts.

Preprocessing:

- JSONPATH: $[?(@.metric == "InterserverConnection")].value.first()

Network errors (timeouts and connection failures) during query execution, background pool tasks and DNS cache update.

Preprocessing:

- JSONPATH: $[?(@.event == "NetworkErrors")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of read (read, pread, io_getevents, etc.) syscalls in fly

Preprocessing:

- JSONPATH: $[?(@.metric == "Read")].value.first()

Number of write (write, pwrite, io_getevents, etc.) syscalls in fly

Preprocessing:

- JSONPATH: $[?(@.metric == "Write")].value.first()

"Total number of bytes allocated by the application."

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.allocated")].value.first()

Maximum number of bytes in physically resident data pages mapped by the allocator,

comprising all pages dedicated to allocator metadata, pages backing active allocations,

and unused dirty pages.

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.resident")].value.first()

"Total number of bytes in active extents mapped by the allocator."

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.mapped")].value.first()

"Total amount of memory (bytes) allocated in currently executing queries."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTracking")].value.first()

"Total amount of memory (bytes) allocated in background processing pool (that is dedicated for background merges, mutations and fetches).

Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa. This happens naturally due to caches for tables indexes and doesn't indicate memory leaks."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundProcessingPool")].value.first()

"Total amount of memory (bytes) allocated in background processing pool (that is dedicated for background moves). Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa.

This happens naturally due to caches for tables indexes and doesn't indicate memory leaks."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundMoveProcessingPool")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

"Total amount of memory (bytes) allocated in background schedule pool (that is dedicated for bookkeeping tasks of Replicated tables)."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundSchedulePool")].value.first()

Total amount of memory (bytes) allocated for background merges. Included in MemoryTrackingInBackgroundProcessingPool. Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa.

This happens naturally due to caches for tables indexes and doesn't indicate memory leaks.

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingForMerges")].value.first()

Number of pending files to process for asynchronous insertion into Distributed tables. Number of files for every shard is summed.

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedFilesToInsert")].value.first()

Connection retries in replicated DB connection pool

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedConnectionFailTry")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Connection failures after all retries in replicated DB connection pool"

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedConnectionFailAtAll")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Maximum replica queue delay relative to current time

Preprocessing:

- JSONPATH: $[?(@.metric == "ReplicasMaxAbsoluteDelay")].value.first()

Preprocessing:

- JSONPATH: $[?(@.metric == "ReplicasSumQueueSize")].value.first()

Number of Replicated tables that are currently in readonly state

due to re-initialization after ZooKeeper session loss

or due to startup without ZooKeeper configured.

Preprocessing:

- JSONPATH: $[?(@.metric == "ReadonlyReplica")].value.first()

Table size in bytes. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].bytes.first()

Number of parts of the table. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].parts.first()

Number of rows in the table. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].rows.first()

Database size in bytes.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}")].bytes.sum()

Whether the replica is in read-only mode.

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when reinitializing sessions in ZooKeeper, and during session reinitialization in ZooKeeper.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].is_readonly.first()

True if the ZooKeeper session expired

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].is_session_expired.first()

Number of data parts that will appear as the result of INSERTs or merges that haven't been done yet.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].future_parts.first()

Number of data parts in the queue for verification. A part is put in the verification queue if there is suspicion that it might be damaged.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].parts_to_check.first()

Size of the queue for operations waiting to be performed.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].queue_size.first()

Number of inserts of blocks of data that need to be made.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].inserts_in_queue.first()

Number of merges waiting to be made.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].merges_in_queue.first()

Maximum entry number in the log of general activity. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].log_max_index.first()

Maximum entry number in the log of general activity that the replica copied to its execution queue, plus one. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].log_pointer.first()

Total number of known replicas of this table. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].total_replicas.first()

Number of replicas of this table that have a session in ZooKeeper (i.e., the number of functioning replicas). (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].active_replicas.first()

Difference between log_max_index and log_pointer

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].replica_lag.first()

The amount of RAM the dictionary uses.

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].bytes_allocated.first()

Number of items stored in the dictionary.

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].element_count.first()

The percentage filled in the dictionary (for a hashed dictionary, the percentage filled in the hash table).

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].bytes_allocated.first()

- MULTIPLIER: 100

Number of sessions (connections) to ZooKeeper. Should be no more than one.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperSession")].value.first()

Number of watches (e.g., event subscriptions) in ZooKeeperr.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperWatch")].value.first()

Number of requests to ZooKeeper in progress.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperRequest")].value.first()

Time spent in waiting for ZooKeeper operations.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperWaitMicroseconds")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- MULTIPLIER: 0.000001

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions that does not belong to user/hardware exceptions.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperOtherExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions caused by session moved/expired, connection loss, marshalling error, operation timed out and invalid zhandle state.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperHardwareExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions caused by no znodes, bad version, node exists, node empty and no children for ephemeral.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperUserExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Get information about the number of events that have occurred in the system.

Preprocessing:

- JSONPATH: $.data

Get metrics which can be calculated instantly, or have a current value format JSONEachRow

Preprocessing:

- JSONPATH: $.data

Get metrics that are calculated periodically in the background

Preprocessing:

- JSONPATH: $.data

Get information about settings that are currently in use.

Preprocessing:

- JSONPATH: $.data

- DISCARD_UNCHANGED_HEARTBEAT: 1h

-

Preprocessing:

- JSONPATH: $.data

-

Preprocessing:

- JSONPATH: $.data

-

Preprocessing:

- JSONPATH: $.data

-

Manual close: YES

-

Manual close: YES

-

Manual close: YES

Depends on:

- ClickHouse: Port {$CLICKHOUSE.PORT} is unavailable

ClickHouse version has changed. Ack to close.

Manual close: YES

Uptime is less than 10 minutes

Manual close: YES

Zabbix has not received data for items for the last 30 minutes

Manual close: YES

Depends on:

- ClickHouse: Service is down

Clickhouse have INSERT queries that are throttled due to high number of active data parts for partition in a MergeTree, please decrease INSERT frequency

Manual close: YES

Descease INSERT queries frequency.

Clickhouse MergeTree table engine split each INSERT query to partitions (PARTITION BY expression)

and add one or more PARTS per INSERT inside each partition,

after that background merge process run, and when you have too much unmerged parts inside partition,

SELECT queries performance can significate degrade, so clickhouse try delay insert, or abort it.

Manual close: YES

Number of errors (timeouts and connection failures) during query execution, background pool tasks and DNS cache update is too high.

"Clickhouse servers and in config.xml

https://clickhouse.tech/docs/en/operations/table_engines/distributed/"

Manual close: YES

When replica have too much lag, it can be skipped from Distributed SELECT Queries without errors

and you will have wrong query results.

Manual close: YES

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when reinitializing sessions in ZooKeeper, and during session reinitialization in ZooKeeper.

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when reinitializing sessions in ZooKeeper, and during session reinitialization in ZooKeeper.

-

-

-

Number of sessions (connections) to ZooKeeper.

Should be no more than one, because using more than one connection to ZooKeeper may lead to bugs due to lack of linearizability (stale reads) that ZooKeeper consistency model allows.

ClickHouse configuration has been changed. Ack to close.

Manual close: YES

Maximum size of distributed files queue to insert for trigger expression.

Maximum number of delayed inserts for trigger expression.

Filter of discoverable databases

Filter to exclude discovered databases

Filter of discoverable dictionaries

Filter to exclude discovered dictionaries

Maximum diff between log_pointer and log_max_index.

Maximum number of smth for trigger expression

Maximum number of parts per partition for trigger expression.

-

The port of ClickHouse HTTP endpoint

Maximum ClickHouse query time in seconds for trigger expression

Maximum size of the queue for operations waiting to be performed for trigger expression.

Replication lag across all tables for trigger expression.

Request scheme which may be http or https

-

Info about tables

Filter:

- A: {#DB} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.MATCHES}

- B: {#DB} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.NOT_MATCHES}

Info about replicas

Filter:

- A: {#DB} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.MATCHES}

- B: {#DB} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DB.NOT_MATCHES}

Info about dictionaries

Filter:

- A: {#NAME} MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DICT.MATCHES}

- B: {#NAME} NOT_MATCHES_REGEX {$CLICKHOUSE.LLD.FILTER.DICT.NOT_MATCHES}

Get longest running query.

-

Preprocessing:

- DISCARD_UNCHANGED_HEARTBEAT: 10m

Preprocessing:

- REGEX: Ok\. 1

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- DISCARD_UNCHANGED_HEARTBEAT: 10m

Version of the server

Preprocessing:

- DISCARD_UNCHANGED_HEARTBEAT: 1d

Revision of the server.

Preprocessing:

- JSONPATH: $[?(@.metric == "Revision")].value.first()

Number of seconds since ClickHouse server start

Preprocessing:

- JSONPATH: $[?(@.metric == "Uptime")].value.first()

Number of queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.data.event == "Query")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of SELECT queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.event == "SelectQuery")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of INSERT queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertQuery")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Number of INSERT queries that are throttled due to high number of active data parts for partition in a MergeTree table."

Preprocessing:

- JSONPATH: $[?(@.metric == "DelayedInserts")].value.first()

Number of executing queries

Preprocessing:

- JSONPATH: $[?(@.metric == "Query")].value.first()

Number of executing background merges

Preprocessing:

- JSONPATH: $[?(@.metric == "Merge")].value.first()

The number of uncompressed bytes inserted in all tables.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Number of bytes (the number of bytes before decompression) read from compressed sources (files, network)."

Preprocessing:

- JSONPATH: $[?(@.event == "ReadCompressedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

The number of rows inserted in all tables.

Preprocessing:

- JSONPATH: $[?(@.event == "InsertedRows")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Rows read for background merges.

Preprocessing:

- JSONPATH: $[?(@.event == "MergedRows")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Uncompressed bytes that were read for background merges

Preprocessing:

- JSONPATH: $[?(@.event == "MergedUncompressedBytes")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Clickhouse MergeTree table engine split each INSERT query to partitions (PARTITION BY expression) and add one or more PARTS per INSERT inside each partition,

after that background merge process run."

Preprocessing:

- JSONPATH: $[?(@.metric == "MaxPartCountForPartition")].value.first()

Number of connections to TCP server (clients with native interface).

Preprocessing:

- JSONPATH: $[?(@.metric == "TCPConnection")].value.first()

Number of connections to HTTP server.

Preprocessing:

- JSONPATH: $[?(@.metric == "HTTPConnection")].value.first()

Number of connections to remote servers sending data that was INSERTed into Distributed tables.

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedSend")].value.first()

Number of connections to MySQL server.

Preprocessing:

- JSONPATH: $[?(@.metric == "MySQLConnection")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

Number of connections from other replicas to fetch parts.

Preprocessing:

- JSONPATH: $[?(@.metric == "InterserverConnection")].value.first()

Network errors (timeouts and connection failures) during query execution, background pool tasks and DNS cache update.

Preprocessing:

- JSONPATH: $[?(@.event == "NetworkErrors")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Number of read (read, pread, io_getevents, etc.) syscalls in fly

Preprocessing:

- JSONPATH: $[?(@.metric == "Read")].value.first()

Number of write (write, pwrite, io_getevents, etc.) syscalls in fly

Preprocessing:

- JSONPATH: $[?(@.metric == "Write")].value.first()

"Total number of bytes allocated by the application."

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.allocated")].value.first()

"Maximum number of bytes in physically resident data pages mapped by the allocator,

comprising all pages dedicated to allocator metadata, pages backing active allocations,

and unused dirty pages."

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.resident")].value.first()

"Total number of bytes in active extents mapped by the allocator."

Preprocessing:

- JSONPATH: $[?(@.metric == "jemalloc.mapped")].value.first()

"Total amount of memory (bytes) allocated in currently executing queries."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTracking")].value.first()

"Total amount of memory (bytes) allocated in background processing pool (that is dedicated for background merges, mutations and fetches).

Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa. This happens naturally due to caches for tables indexes and doesn't indicate memory leaks."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundProcessingPool")].value.first()

"Total amount of memory (bytes) allocated in background processing pool (that is dedicated for background moves). Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa.

This happens naturally due to caches for tables indexes and doesn't indicate memory leaks."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundMoveProcessingPool")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

"Total amount of memory (bytes) allocated in background schedule pool (that is dedicated for bookkeeping tasks of Replicated tables)."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingInBackgroundSchedulePool")].value.first()

"Total amount of memory (bytes) allocated for background merges. Included in MemoryTrackingInBackgroundProcessingPool. Note that this value may include a drift when the memory was allocated in a context of background processing pool and freed in other context or vice-versa.

This happens naturally due to caches for tables indexes and doesn't indicate memory leaks."

Preprocessing:

- JSONPATH: $[?(@.metric == "MemoryTrackingForMerges")].value.first()

Number of pending files to process for asynchronous insertion into Distributed tables. Number of files for every shard is summed.

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedFilesToInsert")].value.first()

Connection retries in replicated DB connection pool

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedConnectionFailTry")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

"Connection failures after all retries in replicated DB connection pool"

Preprocessing:

- JSONPATH: $[?(@.metric == "DistributedConnectionFailAtAll")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Maximum replica queue delay relative to current time

Preprocessing:

- JSONPATH: $[?(@.metric == "ReplicasMaxAbsoluteDelay")].value.first()

Preprocessing:

- JSONPATH: $[?(@.metric == "ReplicasSumQueueSize")].value.first()

"Number of Replicated tables that are currently in readonly state

due to re-initialization after ZooKeeper session loss

or due to startup without ZooKeeper configured."

Preprocessing:

- JSONPATH: $[?(@.metric == "ReadonlyReplica")].value.first()

Table size in bytes. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].bytes.first()

Number of parts of the table. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].parts.first()

Number of rows in the table. Database: {#DB}, table: {#TABLE}

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].rows.first()

Database size in bytes.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}")].bytes.sum()

Whether the replica is in read-only mode.

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when reinitializing sessions in ZooKeeper, and during session reinitialization in ZooKeeper.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].is_readonly.first()

True if the ZooKeeper session expired

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].is_session_expired.first()

Number of data parts that will appear as the result of INSERTs or merges that haven't been done yet.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].future_parts.first()

Number of data parts in the queue for verification. A part is put in the verification queue if there is suspicion that it might be damaged.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].parts_to_check.first()

Size of the queue for operations waiting to be performed.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].queue_size.first()

Number of inserts of blocks of data that need to be made.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].inserts_in_queue.first()

Number of merges waiting to be made.

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].merges_in_queue.first()

Maximum entry number in the log of general activity. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].log_max_index.first()

Maximum entry number in the log of general activity that the replica copied to its execution queue, plus one. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].log_pointer.first()

Total number of known replicas of this table. (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].total_replicas.first()

Number of replicas of this table that have a session in ZooKeeper (i.e., the number of functioning replicas). (Have a non-zero value only where there is an active session with ZooKeeper).

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].active_replicas.first()

Difference between log_max_index and log_pointer

Preprocessing:

- JSONPATH: $[?(@.database == "{#DB}" && @.table == "{#TABLE}")].replica_lag.first()

The amount of RAM the dictionary uses.

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].bytes_allocated.first()

Number of items stored in the dictionary.

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].element_count.first()

The percentage filled in the dictionary (for a hashed dictionary, the percentage filled in the hash table).

Preprocessing:

- JSONPATH: $[?(@.name == "{#NAME}")].bytes_allocated.first()

- MULTIPLIER: 100

Number of sessions (connections) to ZooKeeper. Should be no more than one.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperSession")].value.first()

Number of watches (e.g., event subscriptions) in ZooKeeperr.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperWatch")].value.first()

Number of requests to ZooKeeper in progress.

Preprocessing:

- JSONPATH: $[?(@.metric == "ZooKeeperRequest")].value.first()

Time spent in waiting for ZooKeeper operations.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperWaitMicroseconds")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- MULTIPLIER: 0.000001

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions that does not belong to user/hardware exceptions.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperOtherExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions caused by session moved/expired, connection loss, marshalling error, operation timed out and invalid zhandle state.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperHardwareExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Count of ZooKeeper exceptions caused by no znodes, bad version, node exists, node empty and no children for ephemeral.

Preprocessing:

- JSONPATH: $[?(@.event == "ZooKeeperUserExceptions")].value.first()

⛔️ON_FAIL: CUSTOM_VALUE -> 0

- CHANGE_PER_SECOND

Get information about the number of events that have occurred in the system.

Preprocessing:

- JSONPATH: $.data

Get metrics which can be calculated instantly, or have a current value format JSONEachRow

Preprocessing:

- JSONPATH: $.data

Get metrics that are calculated periodically in the background

Preprocessing:

- JSONPATH: $.data

Get information about settings that are currently in use.

Preprocessing:

- JSONPATH: $.data

- DISCARD_UNCHANGED_HEARTBEAT: 1h

-

Preprocessing:

- JSONPATH: $.data

-

Preprocessing:

- JSONPATH: $.data

-

Preprocessing:

- JSONPATH: $.data

-

Manual close: YES

-

Manual close: YES

-

Manual close: YES

Depends on:

- ClickHouse: Port {$CLICKHOUSE.PORT} is unavailable

ClickHouse version has changed. Ack to close.

Manual close: YES

Uptime is less than 10 minutes.

Manual close: YES

Zabbix has not received data for items for the last 30 minutes

Manual close: YES

Depends on:

- ClickHouse: Service is down

Clickhouse have INSERT queries that are throttled due to high number of active data parts for partition in a MergeTree, please decrease INSERT frequency

Manual close: YES

"Descease INSERT queries frequency.

Clickhouse MergeTree table engine split each INSERT query to partitions (PARTITION BY expression)

and add one or more PARTS per INSERT inside each partition,

after that background merge process run, and when you have too much unmerged parts inside partition,

SELECT queries performance can significate degrade, so clickhouse try delay insert, or abort it"

Manual close: YES

Number of errors (timeouts and connection failures) during query execution, background pool tasks and DNS cache update is too high.

"Clickhouse servers and in config.xml

https://clickhouse.tech/docs/en/operations/table_engines/distributed/"

Manual close: YES

"When replica have too much lag, it can be skipped from Distributed SELECT Queries without errors

and you will have wrong query results."

Manual close: YES

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when reinitializing sessions in ZooKeeper, and during session reinitialization in ZooKeeper.

This mode is turned on if the config doesn't have sections with ZooKeeper, if an unknown error occurred when reinitializing sessions in ZooKeeper, and during session reinitialization in ZooKeeper.

-

-

-

"Number of sessions (connections) to ZooKeeper.

Should be no more than one, because using more than one connection to ZooKeeper may lead to bugs due to lack of linearizability (stale reads) that ZooKeeper consistency model allows."

ClickHouse configuration has been changed. Ack to close.

Manual close: YES