Events and resolution

Overview

There are several types of events generated in Zabbix:

• trigger events - whenever a trigger changes its status (OK→PROBLEM→OK)
• service events - whenever a service changes its status (OK→PROBLEM→OK)
• discovery events - when hosts or services are detected
• autoregistration events - when active agents are auto-registered by server
• internal events - when an item/low-level discovery rule becomes unsupported or a trigger goes into an unknown state

Events are time-stamped and can be the basis of actions such as sending notification email, creating tickets, or executing remote commands.

To view event details in the frontend, go to Monitoring > Problems. There you can click the event date and time to open the event details page, where you can see additional information about the event, including whether it was suppressed because of maintenance or suppressed manually by a user.

Problem resolution

Problem events created by triggers are resolved automatically, if the condition that caused the problem no longer exists.

To add flexibility and convenience to problem event management, the following options exist:

• Event correlation
• Event acknowledgment and suppression
• Manual closing of problems
• Maintenance periods - define periods when problems are suppressed
• Tagging - tag problems by common feature

Trigger events

Change of trigger state is the most frequent and most important source of events. Each time the trigger changes its state, an event is generated. The event contains details of the trigger state's change - when it happened and what the new state is.

Two types of events are created by triggers - Problem and OK.

Problem events

A problem event is created:

• when a trigger expression evaluates to TRUE if the trigger is in OK state;
• each time a trigger expression evaluates to TRUE if multiple problem event generation is enabled for the trigger.

OK events

An OK event closes the related problem event(s) and may be created by 3 components:

• triggers - based on 'OK event generation' and 'OK event closes' settings;
• event correlation
• task manager - when an event is manually closed

Triggers have an 'OK event generation' setting that controls how OK events are generated:

• Expression - an OK event is generated for a trigger in problem state when its expression evaluates to FALSE. This is the simplest setting, enabled by default.
• Recovery expression - an OK event is generated for a trigger in problem state when its expression evaluates to FALSE and the recovery expression evaluates to TRUE. This can be used if trigger recovery criteria is different from problem criteria.
• None - an OK event is never generated. This can be used in conjunction with multiple problem event generation to simply send a notification when something happens.

Additionally triggers have an 'OK event closes' setting that controls which problem events are closed:

• All problems - an OK event will close all open problems created by the trigger
• All problems if tag values match - an OK event will close open problems created by the trigger and having at least one matching tag value. The tag is defined by 'Tag for matching' trigger setting. If there are no problem events to close then OK event is not generated. This is often called trigger level event correlation.

Event correlation

Event correlation (also called global event correlation) is a way to set up custom event closing (resulting in OK event generation) rules.

The rules define how the new problem events are paired with existing problem events and allow to close the new event or the matched events by generating corresponding OK events.

However, event correlation must be configured very carefully, as it can negatively affect event processing performance or, if misconfigured, close more events than intended (in the worst case even all problem events could be closed). A few configuration tips:

1. always reduce the correlation scope by setting a unique tag for the control event (the event that is paired with old events) and use the 'new event tag' correlation condition
2. don't forget to add a condition based on the old event when using 'close old event' operation, or all existing problems could be closed
3. avoid using common tag names used by different correlation configurations

Task manager

If the 'Allow manual close' setting is enabled for trigger, then it's possible to manually close problem events generated by the trigger. This is done in the frontend when updating a problem. The event is not closed directly - instead a 'close event' task is created, which is handled by the task manager shortly. The task manager will generate a corresponding OK event and the problem event will be closed.

Service events

Service events are generated only if service actions for these events are enabled. In this case, each change of service state creates a new event:

• Problem event - when service state is changed from OK to Problem
• OK event - when service state is changed from Problem to OK

The event contains details of the service state change - when it happened and what the new state is.

Discovery events

Zabbix periodically scans the IP ranges defined in network discovery rules. Frequency of the check is configurable for each rule individually. Once a host or a service is discovered, a discovery event (or several events) are generated.

Zabbix generates the following events:

Active agent autoregistration events

Active agent autoregistration creates events in Zabbix.

If configured, active agent autoregistration event is created when a previously unknown active agent asks for checks or if the host metadata has changed. The server adds a new auto-registered host, using the received IP address and port of the agent.

For more information, see the active agent autoregistration page.

Internal events

Internal events happen when:

• an item changes state from 'normal' to 'unsupported'
• an item changes state from 'unsupported' to 'normal'
• a low-level discovery rule changes state from 'normal' to 'unsupported'
• a low-level discovery rule changes state from 'unsupported' to 'normal'
• a trigger changes state from 'normal' to 'unknown'
• a trigger changes state from 'unknown' to 'normal'

The aim of introducing internal events is to allow users to be notified when any internal event takes place, for example, an item becomes unsupported and stops gathering data.

Internal events are only created when internal actions for these events are enabled. To stop generation of internal events (for example, for items becoming unsupported), disable all actions for internal events in Alerts > Actions > Internal actions.

See also: Receiving notification on unsupported items