Elasticsearch setup

Zabbix can store history data in Elasticsearch as an alternative to a relational database.

This guide covers the setup for the supported versions of Elasticsearch. Zabbix server does not start if your Elasticsearch version is older than the minimum supported version. If it is newer than the maximum supported version, Zabbix server starts only if the AllowUnsupportedDBVersions parameter is set to 1.

Elasticsearch can store the following value types:

Important notes

• Elasticsearch requires libcurl. See requirements for details.
• The housekeeper does not delete data from Elasticsearch. To control how long data is kept, see Index Lifecycle Management.
• Zabbix does not calculate or store trends in Elasticsearch. Consider extending the history storage period to preserve older data.
• When Elasticsearch is used, range queries retrieving values from the database are limited by the timestamp of the data storage period.
• Elasticsearch is not supported for Zabbix proxies.
• Starting with Elasticsearch 8.0, security features are enabled by default: Elasticsearch requires authentication and uses HTTPS for its HTTP API, with certificates signed by an automatically generated CA. The examples on this page use http://localhost:9200 without authentication. If security is enabled, use an https:// URL and pass the CA certificate and credentials in your requests (for example, sudo curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://localhost:9200; the CA certificate file can be read only by root and the elasticsearch group). To connect Zabbix server and Zabbix frontend to such a cluster, see Connecting to a secured cluster.

If Elasticsearch isn't installed yet, refer to the official installation guide before proceeding.

Configuring Elasticsearch

To store history data in Elasticsearch, you need to:

• Create an index for each value type you want to store—this is where Elasticsearch stores the data, similar to a table in a relational database.
• Define a mapping for each index—this defines the structure of the data, similar to a table schema.
• Set up an ingest pipeline to process values before storage (required for JSON values and date-based indices).

Elasticsearch can store data in a single index per value type, or across multiple date-based indices. Both approaches are described below.

Storing history in a single index

In this approach, all history data for a given value type is written to a single index (e.g., uint or text).

To create an index for the Numeric (unsigned) value type, send the following request (with /uint in the URL) to your Elasticsearch instance:

curl -X PUT \
 http://localhost:9200/uint \
 -H 'content-type:application/json' \
 -d '{
   "settings": {
      "index": {
         "number_of_replicas": 1,
         "number_of_shards": 5
      }
   },
   "mappings": {
      "properties": {
         "itemid": { "type": "long" },
         "clock": { "format": "epoch_second", "type": "date" },
         "value": { "type": "long" }
      }
   }
}'

Elasticsearch will respond with a confirmation that the index was created:

{"acknowledged": true, "shards_acknowledged": true, "index": "uint"}

Similar requests must be sent for each value type you want to store in Elasticsearch, that is, for each type listed in the value_types option of the Zabbix server configuration.

For example, to create an index for the Text value type:

curl -X PUT \
 http://localhost:9200/text \
 -H 'content-type:application/json' \
 -d '{
   "settings": {
      "index": {
         "number_of_replicas": 1,
         "number_of_shards": 5
      }
   },
   "mappings": {
      "properties": {
         "itemid": { "type": "long" },
         "clock": { "format": "epoch_second", "type": "date" },
         "value": {
            "fields": {
               "analyzed": { "index": true, "type": "text", "analyzer": "standard" }
            },
            "index": false,
            "type": "text"
         }
      }
   }
}'

JSON value type

Unlike other value types, JSON values require additional processing before storage. The index for the JSON value type stores each value in one of two fields:

• value_parsed—JSON objects, stored as a flattened field, so that they can be searched by key and value;
• value_raw—all other JSON values (arrays, strings, numbers, booleans, and null), stored as a string.

An ingest pipeline parses each value as JSON and stores it in the correct field. Values that are not valid JSON are not sent to Elasticsearch, because the items that receive them become unsupported.

First, create the ingest pipeline:

curl -X PUT \
 http://localhost:9200/_ingest/pipeline/json \
 -H 'content-type:application/json' \
 -d '{
   "processors": [
      {
         "json": {
            "field": "value",
            "target_field": "value_parsed",
            "ignore_failure": true
         }
      },
      {
         "script": {
            "source": "if (ctx.value_parsed == null || !(ctx.value_parsed instanceof Map)) { ctx.value_raw = ctx.value; ctx.remove(\"value_parsed\"); }"
         }
      }
   ]
}'

Elasticsearch will respond with a confirmation that the ingest pipeline was created:

{"acknowledged": true}

Then, create an index for the JSON value type by sending the following request (with /json in the URL) to your Elasticsearch instance. The default_pipeline setting makes Elasticsearch process every value written to this index with the json ingest pipeline. This setting is required, because Zabbix server does not specify an ingest pipeline when storing history in a single index.

curl -X PUT \
 http://localhost:9200/json \
 -H 'content-type:application/json' \
 -d '{
   "settings": {
      "index": {
         "number_of_replicas": 1,
         "number_of_shards": 5,
         "default_pipeline": "json"
      }
   },
   "mappings": {
      "dynamic": false,
      "properties": {
         "itemid": { "type": "long" },
         "clock": { "type": "date", "format": "epoch_second" },
         "ns": { "type": "long" },
         "value_parsed": { "type": "flattened", "ignore_above": 8191 },
         "value_raw": { "type": "keyword", "ignore_above": 8191 }
      }
   }
}'

Storing history in date-based indices

Instead of writing all history data to a single index (e.g., uint), Elasticsearch can distribute this data across multiple date-based indices (e.g., uint-2026-01-01, uint-2026-01-02). This makes it easier to manage data volume and retention over time.

To enable this, you need to:

• Create an index template for each value type you want to store—this tells Elasticsearch what mapping to apply when it automatically creates a new date-based index.
• Create an ingest pipeline for each value type—it processes each incoming value and routes it to the correct date-based index.

Index templates

To create a template for the text index, send a request with the following details:

• Use _index_template/text_template in the URL of your Elasticsearch instance.
• Use "text-*" in the "index_patterns" field to match the names of date-based indices.
• In the "template" field, specify the index settings and a mapping for the text value type (see mappings in the Zabbix source repository).

curl -X PUT \
 http://localhost:9200/_index_template/text_template \
 -H 'content-type:application/json' \
 -d '{
   "index_patterns": [ "text-*" ],
   "template": {
      "settings": {
         "index": {
            "number_of_replicas": 1,
            "number_of_shards": 5
         }
      },
      "mappings": {
         "properties": {
            "itemid": { "type": "long" },
            "clock": { "format": "epoch_second", "type": "date" },
            "value": {
               "fields": {
                  "analyzed": { "index": true, "type": "text", "analyzer": "standard" }
               },
               "index": false,
               "type": "text"
            }
         }
      }
   }
}'

Create a template for each value type you want to store, changing the template name, the index pattern, and the mapping accordingly.

For example, template for the json index (for details on the value_parsed and value_raw fields, see JSON value type):

curl -X PUT \
 http://localhost:9200/_index_template/json_template \
 -H 'content-type:application/json' \
 -d '{
   "index_patterns": [ "json-*" ],
   "template": {
      "settings": {
         "number_of_replicas": 1
      },
      "mappings": {
         "dynamic": false,
         "properties": {
            "itemid": { "type": "long" },
            "clock": { "type": "date", "format": "epoch_second" },
            "ns": { "type": "long" },
            "value_parsed": { "type": "flattened", "ignore_above": 8191 },
            "value_raw": { "type": "keyword", "ignore_above": 8191 }
         }
      }
   }
}'

Ingest pipelines

To create an ingest pipeline for the text index:

• Use _ingest/pipeline/text-pipeline in the URL of your Elasticsearch instance.
• Include a date_index_name processor to route each value to the correct date-based index based on its timestamp.

curl -X PUT \
 http://localhost:9200/_ingest/pipeline/text-pipeline \
 -H 'content-type:application/json' \
 -d '{
   "description": "daily text index naming",
   "processors": [
      {
         "date_index_name": {
            "field": "clock",
            "date_formats": ["UNIX"],
            "index_name_prefix": "text-",
            "date_rounding": "d"
         }
      }
   ]
}'

Create a pipeline for each value type you want to store, changing the pipeline name, the description, and the "index_name_prefix" value accordingly. The pipeline name must consist of the index name and the -pipeline suffix (for example, uint-pipeline), because Zabbix server uses this name when sending values to Elasticsearch.

For the json index, the pipeline also needs to parse the JSON value before routing it to the correct index:

curl -X PUT \
 http://localhost:9200/_ingest/pipeline/json-pipeline \
 -H 'content-type:application/json' \
 -d '{
   "description": "daily json index naming",
   "processors": [
      {
         "json": {
            "field": "value",
            "target_field": "value_parsed",
            "ignore_failure": true
         }
      },
      {
         "script": {
            "source": "if (ctx.value_parsed == null || !(ctx.value_parsed instanceof Map)) { ctx.value_raw = ctx.value; ctx.remove(\"value_parsed\"); }"
         }
      },
      {
         "date_index_name": {
            "field": "clock",
            "date_formats": [ "UNIX" ],
            "index_name_prefix": "json-",
            "date_rounding": "d"
         }
      }
   ]
}'

Configuring Zabbix server

In your Zabbix server configuration file (zabbix_server.conf), set the HistoryProvider parameter.

For example, to store Character, Log, Text, and JSON type values in Elasticsearch (while keeping Numeric values in a database):

HistoryProvider=elasticsearch;value_types="str,log,text,json",url=http://localhost:9200

If you're using date-based indices, add date_index=1 to the parameter:

HistoryProvider=elasticsearch;value_types="str,log,text,json",url=http://localhost:9200,date_index=1

If Elasticsearch security features are enabled, specify an https:// URL with credentials (see Connecting to a secured cluster).

After making changes, restart Zabbix server:

systemctl restart zabbix-server

Configuring Zabbix frontend

In your Zabbix frontend configuration file (zabbix.conf.php), set the $HISTORY_PROVIDERS variable to match the server configuration:

$HISTORY_PROVIDERS[] = [
  'types' => ['str','log','text','json'],
  'provider' => 'elasticsearch',
  'url' => 'http://localhost:9200'
];

If Elasticsearch security features are enabled, specify the same https:// URL with credentials as in the server configuration (see Connecting to a secured cluster).

Connecting to a secured cluster

If Elasticsearch security features are enabled (default since Elasticsearch 8.0), Zabbix server and Zabbix frontend must trust the CA certificate of the Elasticsearch HTTP layer and must authenticate with a username and password.

1. Create a role and a user for Zabbix. The following role allows Zabbix to retrieve the Elasticsearch version (the monitor cluster privilege), create indices, write and read history data, and delete history data when item history is cleared:

sudo curl -X PUT \
 --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic \
 https://localhost:9200/_security/role/zabbix_history \
 -H 'content-type:application/json' \
 -d '{
   "cluster": [ "monitor" ],
   "indices": [
      {
         "names": [ "uint", "uint-*", "dbl", "dbl-*", "str", "str-*", "log", "log-*", "text", "text-*", "json", "json-*" ],
         "privileges": [ "create_index", "index", "read", "delete" ]
      }
   ]
}'

sudo curl -X PUT \
 --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic \
 https://localhost:9200/_security/user/zabbix \
 -H 'content-type:application/json' \
 -d '{
   "password": "<password>",
   "roles": [ "zabbix_history" ]
}'

2. Add the Elasticsearch CA certificate to the trusted certificates of the operating system on the Zabbix server and Zabbix frontend hosts. If Zabbix runs on a different host than Elasticsearch, copy the CA certificate (/etc/elasticsearch/certs/http_ca.crt) to that host first. For example, on Debian/Ubuntu:

sudo cp /etc/elasticsearch/certs/http_ca.crt /usr/local/share/ca-certificates/elasticsearch-http-ca.crt
sudo update-ca-certificates

The host name in the URL must match the Elasticsearch HTTP certificate. The automatically generated certificate is valid for localhost, the host name, and the IP addresses of the Elasticsearch host.

3. Specify the credentials in the URL, both in the Zabbix server configuration file and in the frontend configuration file. If the password contains special characters, percent-encode them (for example, @ as %40, : as %3A, and , as %2C).

HistoryProvider=elasticsearch;value_types="str,log,text,json",url=https://zabbix:<password>@localhost:9200

$HISTORY_PROVIDERS[] = [
  'types' => ['str','log','text','json'],
  'provider' => 'elasticsearch',
  'url' => 'https://zabbix:<password>@localhost:9200'
];

4. Restart Zabbix server and PHP-FPM (or the web server, if PHP runs as a web server module).

Troubleshooting

The following steps may help you troubleshoot problems with your Elasticsearch setup:

1. Check Elasticsearch or Zabbix server logs for errors. For example, the following Zabbix server log messages indicate problems with connecting to a secured cluster (see Connecting to a secured cluster):

• cannot query Elasticsearch: SSL certificate problem: self-signed certificate in certificate chain—the Elasticsearch CA certificate is not trusted;
• cannot query Elasticsearch, HTTP error message: The requested URL returned error: 401—the credentials are missing or incorrect;
• cannot query Elasticsearch, HTTP error message: The requested URL returned error: 403—the user does not have the required privileges.

Zabbix frontend displays similar errors (for example, Elasticsearch error: missing authentication credentials for REST request).

2. To identify slow queries, set the LogSlowQueries Zabbix server configuration parameter. If reading history from Elasticsearch takes longer than the specified time, Zabbix server logs slow query: <N> sec, "<query>". While waiting for an Elasticsearch response, Zabbix server also logs waiting for Elasticsearch response <N>sec each time the specified time passes, or every 10 seconds if LogSlowQueries is set to 0.

3. Verify that Elasticsearch allows access from Zabbix server and Zabbix frontend.

4. Verify that auto_create_index is enabled (it is enabled by default):

curl -X GET \
 "http://localhost:9200/_cluster/settings?include_defaults=true&filter_path=**.auto_create_index"

# {"persistent": {"action": {"auto_create_index": "false"} } }

To enable it, send the following request:

curl -X PUT \
 http://localhost:9200/_cluster/settings \
 -H 'content-type:application/json' \
 -d '{
   "persistent": {
      "action.auto_create_index": "true"
   }
}'

# {"acknowledged": true, "persistent": {"action": {"auto_create_index": "true"} }, "transient": {} }

5. Verify that mappings, templates, and ingest pipelines are correct by sending GET requests to their respective URLs:

curl -X GET "http://localhost:9200/json*"
curl -X GET "http://localhost:9200/_index_template/json*"
curl -X GET "http://localhost:9200/_ingest/pipeline/json*"

You can compare received responses with expected responses in the Elasticsearch API documentation.

6. Check if any shards are in a failed state; restarting Elasticsearch may resolve this. On a single-node cluster, indices with replicas ("number_of_replicas": 1) have the yellow status, because replicas cannot be allocated to the same node as primary shards; this does not affect Zabbix. To avoid it, set "number_of_replicas" to 0.

7. Query Elasticsearch to see if the data collected by Zabbix is stored, for example:

curl "http://localhost:9200/json*/_search" \
 -H 'Content-Type: application/json' \
 -d '{
   "query": {
     "term": {
       "itemid": 42269
     }
   }
 }'

8. If you need to reset your Elasticsearch setup and start over, delete the indices, index templates, and ingest pipelines created for Zabbix (this deletes all history data stored in Elasticsearch). By default, Elasticsearch does not allow deleting indices by wildcard expressions, so to delete date-based indices, temporarily change the action.destructive_requires_name setting. Index templates and ingest pipelines must be deleted one by one; requests for templates or pipelines that do not exist return an error, which can be ignored.

curl -X DELETE "http://localhost:9200/uint,dbl,str,log,text,json?ignore_unavailable=true"

curl -X PUT \
 http://localhost:9200/_cluster/settings \
 -H 'content-type:application/json' \
 -d '{"persistent": {"action.destructive_requires_name": false}}'
curl -X DELETE "http://localhost:9200/uint-*,dbl-*,str-*,log-*,text-*,json-*"
curl -X PUT \
 http://localhost:9200/_cluster/settings \
 -H 'content-type:application/json' \
 -d '{"persistent": {"action.destructive_requires_name": null}}'

for type in uint dbl str log text json; do
  curl -X DELETE "http://localhost:9200/_index_template/${type}_template"
  curl -X DELETE "http://localhost:9200/_ingest/pipeline/${type}-pipeline"
done
curl -X DELETE "http://localhost:9200/_ingest/pipeline/json"