ZooKeeper

ZooKeeper

Apache ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.

Available solutions




This template is for Zabbix version: 6.4
Also available for: 6.2 6.0 5.4 5.0

Source: https://git.zabbix.com/projects/ZBX/repos/zabbix/browse/templates/app/zookeeper_http?at=release/6.4

Zookeeper by HTTP

Overview

This template is designed for the effortless deployment of Zookeeper monitoring by Zabbix via HTTP and doesn't require any external scripts.

Requirements

Zabbix version: 6.4 and higher.

Tested versions

This template has been tested on:

  • Apache Zookeeper, version 3.6+, 3.8+

Configuration

Zabbix should be configured according to the instructions in the Templates out of the box section.

Setup

This template works with standalone and cluster instances. Metrics are collected from each Zookeeper node by requests to AdminServer. By default AdminServer is enabled and listens on port 8080. You can enable or configure AdminServer parameters according official documentations. Don't forget to change macros {$ZOOKEEPER.COMMAND_URL}, {$ZOOKEEPER.PORT}, {$ZOOKEEPER.SCHEME}.

Macros used

Name Description Default
{$ZOOKEEPER.PORT}

The port the embedded Jetty server listens on (admin.serverPort).

 8080
{$ZOOKEEPER.COMMAND_URL}

The URL for listing and issuing commands relative to the root URL (admin.commandURL).

 commands
{$ZOOKEEPER.SCHEME}

Request scheme which may be http or https

 http
{$ZOOKEEPER.FILE_DESCRIPTORS.MAX.WARN}

Maximum percentage of file descriptors usage alert threshold (for trigger expression).

 85
{$ZOOKEEPER.OUTSTANDING_REQ.MAX.WARN}

Maximum number of outstanding requests (for trigger expression).

 10
{$ZOOKEEPER.PENDING_SYNCS.MAX.WARN}

Maximum number of pending syncs from the followers (for trigger expression).

 10

Items

Name Description Type Key and additional info
Zookeeper: Get server metrics HTTP agent zookeeper.get_metrics
Zookeeper: Get connections stats

Get information on client connections to server. Note, depending on the number of client connections this operation may be expensive (i.e. impact server performance).

 HTTP agent zookeeper.get_connections_stats
Zookeeper: Server mode

Mode of the server. In an ensemble, this may either be leader or follower. Otherwise, it is standalone

 Dependent item zookeeper.server_state

Preprocessing

  • JSON Path: $.server_state

  • Discard unchanged with heartbeat: 1h
Zookeeper: Uptime

Uptime that a peer has been in a table leading/following/observing state.

 Dependent item zookeeper.uptime

Preprocessing

  • JSON Path: $.uptime

  • Custom multiplier: 0.001
Zookeeper: Version

Version of Zookeeper server.

 Dependent item zookeeper.version

Preprocessing

  • JSON Path: $.version

  • Regular expression: ^([0-9\.]+) \1

  • Discard unchanged with heartbeat: 3h
Zookeeper: Approximate data size

Data tree size in bytes.The size includes the znode path and its value.

 Dependent item zookeeper.approximate_data_size

Preprocessing

  • JSON Path: $.approximate_data_size
Zookeeper: File descriptors, max

Maximum number of file descriptors that a zookeeper server can open.

 Dependent item zookeeper.max_file_descriptor_count

Preprocessing

  • JSON Path: $.max_file_descriptor_count

  • Discard unchanged with heartbeat: 1h
Zookeeper: File descriptors, open

Number of file descriptors that a zookeeper server has open.

 Dependent item zookeeper.open_file_descriptor_count

Preprocessing

  • JSON Path: $.open_file_descriptor_count
Zookeeper: Outstanding requests

The number of queued requests when the server is under load and is receiving more sustained requests than it can process.

 Dependent item zookeeper.outstanding_requests

Preprocessing

  • JSON Path: $.outstanding_requests
Zookeeper: Commit per sec

The number of commits performed per second

 Dependent item zookeeper.commit_count.rate

Preprocessing

  • JSON Path: $.commit_count

  • Change per second
Zookeeper: Diff syncs per sec

Number of diff syncs performed per second

 Dependent item zookeeper.diff_count.rate

Preprocessing

  • JSON Path: $.diff_count

  • Change per second
Zookeeper: Snap syncs per sec

Number of snap syncs performed per second

 Dependent item zookeeper.snap_count.rate

Preprocessing

  • JSON Path: $.snap_count

  • Change per second
Zookeeper: Looking per sec

Rate of transitions into looking state.

 Dependent item zookeeper.looking_count.rate

Preprocessing

  • JSON Path: $.looking_count

  • Change per second
Zookeeper: Alive connections

Number of active clients connected to a zookeeper server.

 Dependent item zookeeper.num_alive_connections

Preprocessing

  • JSON Path: $.num_alive_connections
Zookeeper: Global sessions

Number of global sessions.

 Dependent item zookeeper.global_sessions

Preprocessing

  • JSON Path: $.global_sessions
Zookeeper: Local sessions

Number of local sessions.

 Dependent item zookeeper.local_sessions

Preprocessing

  • JSON Path: $.local_sessions
Zookeeper: Drop connections per sec

Rate of connection drops.

 Dependent item zookeeper.connection_drop_count.rate

Preprocessing

  • JSON Path: $.connection_drop_count

  • Change per second
Zookeeper: Rejected connections per sec

Rate of connection rejected.

 Dependent item zookeeper.connection_rejected.rate

Preprocessing

  • JSON Path: $.connection_rejected

  • Change per second
Zookeeper: Revalidate connections per sec

Rate of connection revalidations.

 Dependent item zookeeper.connection_revalidate_count.rate

Preprocessing

  • JSON Path: $.connection_revalidate_count

  • Change per second
Zookeeper: Revalidate per sec

Rate of revalidations.

 Dependent item zookeeper.revalidate_count.rate

Preprocessing

  • JSON Path: $.revalidate_count

  • Change per second
Zookeeper: Latency, max

The maximum amount of time it takes for the server to respond to a client request.

 Dependent item zookeeper.max_latency

Preprocessing

  • JSON Path: $.max_latency
Zookeeper: Latency, min

The minimum amount of time it takes for the server to respond to a client request.

 Dependent item zookeeper.min_latency

Preprocessing

  • JSON Path: $.min_latency
Zookeeper: Latency, avg

The average amount of time it takes for the server to respond to a client request.

 Dependent item zookeeper.avg_latency

Preprocessing

  • JSON Path: $.avg_latency
Zookeeper: Znode count

The number of znodes in the ZooKeeper namespace (the data)

 Dependent item zookeeper.znode_count

Preprocessing

  • JSON Path: $.znode_count

  • Discard unchanged with heartbeat: 1h
Zookeeper: Ephemeral nodes count

Number of ephemeral nodes that a zookeeper server has in its data tree.

 Dependent item zookeeper.ephemerals_count

Preprocessing

  • JSON Path: $.ephemerals_count
Zookeeper: Watch count

Number of watches currently set on the local ZooKeeper process.

 Dependent item zookeeper.watch_count

Preprocessing

  • JSON Path: $.watch_count
Zookeeper: Packets sent per sec

The number of zookeeper packets sent from a server per second.

 Dependent item zookeeper.packets_sent

Preprocessing

  • JSON Path: $.packets_sent

  • Change per second
Zookeeper: Packets received per sec

The number of zookeeper packets received by a server per second.

 Dependent item zookeeper.packets_received.rate

Preprocessing

  • JSON Path: $.packets_received

  • Change per second
Zookeeper: Bytes received per sec

Number of bytes received per second.

 Dependent item zookeeper.bytes_received_count.rate

Preprocessing

  • JSON Path: $.bytes_received_count

  • Change per second
Zookeeper: Election time, avg

Time between entering and leaving election.

 Dependent item zookeeper.avg_election_time

Preprocessing

  • JavaScript: The text is too long. Please see the template.
Zookeeper: Elections

Number of elections happened.

 Dependent item zookeeper.cnt_election_time

Preprocessing

  • JavaScript: The text is too long. Please see the template.
Zookeeper: Fsync time, avg

Time to fsync transaction log.

 Dependent item zookeeper.avg_fsynctime

Preprocessing

  • JavaScript: The text is too long. Please see the template.
Zookeeper: Fsync

Count of performed fsyncs.

 Dependent item zookeeper.cnt_fsynctime

Preprocessing

  • JavaScript: The text is too long. Please see the template.
Zookeeper: Snapshot write time, avg

Average time to write a snapshot.

 Dependent item zookeeper.avg_snapshottime

Preprocessing

  • JavaScript: The text is too long. Please see the template.
Zookeeper: Snapshot writes

Count of performed snapshot writes.

 Dependent item zookeeper.cnt_snapshottime

Preprocessing

  • JavaScript: The text is too long. Please see the template.

Triggers

Name Description Expression Severity Dependencies and additional info
Zookeeper: Server mode has changed

Zookeeper node state has changed. Acknowledge to close the problem manually.

 last(/Zookeeper by HTTP/zookeeper.server_state,#1)<>last(/Zookeeper by HTTP/zookeeper.server_state,#2) and length(last(/Zookeeper by HTTP/zookeeper.server_state))>0 Info Manual close: Yes
Zookeeper: Failed to fetch info data

Zabbix has not received data for items for the last 10 minutes

 nodata(/Zookeeper by HTTP/zookeeper.uptime,10m)=1 Warning Manual close: Yes
Zookeeper: Version has changed

Zookeeper version has changed. Acknowledge to close the problem manually.

 last(/Zookeeper by HTTP/zookeeper.version,#1)<>last(/Zookeeper by HTTP/zookeeper.version,#2) and length(last(/Zookeeper by HTTP/zookeeper.version))>0 Info Manual close: Yes
Zookeeper: Too many file descriptors used

Number of file descriptors used more than {$ZOOKEEPER.FILE_DESCRIPTORS.MAX.WARN}% of the available number of file descriptors.

 min(/Zookeeper by HTTP/zookeeper.open_file_descriptor_count,5m) * 100 / last(/Zookeeper by HTTP/zookeeper.max_file_descriptor_count) > {$ZOOKEEPER.FILE_DESCRIPTORS.MAX.WARN} Warning
Zookeeper: Too many queued requests

Number of queued requests in the server. This goes up when the server receives more requests than it can process.

 min(/Zookeeper by HTTP/zookeeper.outstanding_requests,5m)>{$ZOOKEEPER.OUTSTANDING_REQ.MAX.WARN} Average Manual close: Yes

LLD rule Leader metrics discovery

Name Description Type Key and additional info
Leader metrics discovery

Additional metrics for leader node

 Dependent item zookeeper.metrics.leader

Preprocessing

  • JSON Path: $.server_state

  • JavaScript: The text is too long. Please see the template.

Item prototypes for Leader metrics discovery

Name Description Type Key and additional info
Zookeeper: Pending syncs{#SINGLETON}

Number of pending syncs to carry out to ZooKeeper ensemble followers.

 Dependent item zookeeper.pending_syncs[{#SINGLETON}]

Preprocessing

  • JSON Path: $.pending_syncs
Zookeeper: Quorum size{#SINGLETON} Dependent item zookeeper.quorum_size[{#SINGLETON}]

Preprocessing

  • JSON Path: $.quorum_size
Zookeeper: Synced followers{#SINGLETON}

Number of synced followers reported when a node server_state is leader.

 Dependent item zookeeper.synced_followers[{#SINGLETON}]

Preprocessing

  • JSON Path: $.synced_followers
Zookeeper: Synced non-voting follower{#SINGLETON}

Number of synced voting followers reported when a node server_state is leader.

 Dependent item zookeeper.synced_non_voting_followers[{#SINGLETON}]

Preprocessing

  • JSON Path: $.synced_non_voting_followers
Zookeeper: Synced observers{#SINGLETON}

Number of synced observers.

 Dependent item zookeeper.synced_observers[{#SINGLETON}]

Preprocessing

  • JSON Path: $.synced_observers
Zookeeper: Learners{#SINGLETON}

Number of learners.

 Dependent item zookeeper.learners[{#SINGLETON}]

Preprocessing

  • JSON Path: $.learners

Trigger prototypes for Leader metrics discovery

Name Description Expression Severity Dependencies and additional info
Zookeeper: Too many pending syncs min(/Zookeeper by HTTP/zookeeper.pending_syncs[{#SINGLETON}],5m)>{$ZOOKEEPER.PENDING_SYNCS.MAX.WARN} Average Manual close: Yes
Zookeeper: Too few active followers

The number of followers should equal the total size of your ZooKeeper ensemble, minus 1 (the leader is not included in the follower count). If the ensemble fails to maintain quorum, all automatic failover features are suspended.

 last(/Zookeeper by HTTP/zookeeper.synced_followers[{#SINGLETON}]) < last(/Zookeeper by HTTP/zookeeper.quorum_size[{#SINGLETON}])-1 Average

LLD rule Clients discovery

Name Description Type Key and additional info
Clients discovery

Get list of client connections.

Note, depending on the number of client connections this operation may be expensive (i.e. impact server performance).

 HTTP agent zookeeper.clients

Preprocessing

  • JavaScript: The text is too long. Please see the template.

Item prototypes for Clients discovery

Name Description Type Key and additional info
Zookeeper client {#TYPE} [{#CLIENT}]: Get client info

The item gets information about "{#CLIENT}" client of "{#TYPE}" type.

 Dependent item zookeeper.client_info[{#TYPE},{#CLIENT}]

Preprocessing

  • JSON Path: The text is too long. Please see the template.

    ⛔️Custom on fail: Discard value
Zookeeper client {#TYPE} [{#CLIENT}]: Latency, max

The maximum amount of time it takes for the server to respond to a client request.

 Dependent item zookeeper.max_latency[{#TYPE},{#CLIENT}]

Preprocessing

  • JSON Path: $.max_latency
Zookeeper client {#TYPE} [{#CLIENT}]: Latency, min

The minimum amount of time it takes for the server to respond to a client request.

 Dependent item zookeeper.min_latency[{#TYPE},{#CLIENT}]

Preprocessing

  • JSON Path: $.min_latency
Zookeeper client {#TYPE} [{#CLIENT}]: Latency, avg

The average amount of time it takes for the server to respond to a client request.

 Dependent item zookeeper.avg_latency[{#TYPE},{#CLIENT}]

Preprocessing

  • JSON Path: $.avg_latency
Zookeeper client {#TYPE} [{#CLIENT}]: Packets sent per sec

The number of packets sent.

 Dependent item zookeeper.packets_sent[{#TYPE},{#CLIENT}]

Preprocessing

  • JSON Path: $.packets_sent

  • Change per second
Zookeeper client {#TYPE} [{#CLIENT}]: Packets received per sec

The number of packets received.

 Dependent item zookeeper.packets_received[{#TYPE},{#CLIENT}]

Preprocessing

  • JSON Path: $.packets_received

  • Change per second
Zookeeper client {#TYPE} [{#CLIENT}]: Outstanding requests

The number of queued requests when the server is under load and is receiving more sustained requests than it can process.

 Dependent item zookeeper.outstanding_requests[{#TYPE},{#CLIENT}]

Preprocessing

  • JSON Path: $.outstanding_requests

Feedback

Please report any issues with the template at https://support.zabbix.com

You can also provide feedback, discuss the template, or ask for help at ZABBIX forums

Articles and documentation

+ Propose new article

No se encuentra la integración que necesitas?

Request custom integration

Zabbix integration team will develop custom integration based on your requirements and Zabbix best practices.

Request

Propose your integration

Have you already developed high quality integration and want to submit to Zabbix integration repository?

Propose