1 Trigger examples

This page is a collection of trigger examples.

Examples are sorted by use case:

• Detect unavailable host
• Detect unavailable proxy
• Detect unavailable SMTP cluster
• Detect unreachable host
• Detect unexpected reboot
• Detect changes in important files
• Detect changes in DNS query
• Detect non-matching software on different hosts
• Detect unsynchronized clocks
• Detect outdated agent
• Detect flapping interface
• Detect high incoming traffic
• Detect low disk space (with hysteresis)
• Detect low disk space (adaptive threshold)
• Detect high CPU load
• Compare CPU loads
• Compare long-term CPU loads
• Detect string containing special characters

Detect unavailable host

max(/host/zabbix[host,agent,available],5m)=0

This trigger fires if Zabbix agent on the host has been unavailable for 5 minutes.
Data item: zabbix[host,agent,available]

Alternative method:

nodata(/host/agent.ping,5m)=1

This trigger fires if no data has been received from Zabbix agent for 5 minutes.
Data item: agent.ping

Detect unavailable proxy

fuzzytime(/host/zabbix[proxy,{$PROXY_NAME},lastaccess],1m)=0

This trigger fires if Zabbix proxy data lags behind Zabbix server time by 1 minute.
Data item: zabbix[proxy,{$PROXY_NAME},lastaccess]

Detect unavailable SMTP cluster

last(/smtp1.example.com/net.tcp.service[smtp])=0 and last(/smtp2.example.com/net.tcp.service[smtp])=0

This trigger fires if both SMTP servers are down.
Data item: net.tcp.service

Detect unreachable host

count(/host/icmpping,30m,,"0")>5

This trigger fires if the host is unreachable by ping more than 5 times in the last 30 minutes.
Data item: icmpping

Detect unexpected reboot

change(/host/system.uptime)<0

This trigger fires if a negative change in the system uptime value is encountered (indicating a reboot).
Data item: system.uptime

Detect changes in important files

last(/host/vfs.file.cksum[/etc/passwd],#1)<>last(/host/vfs.file.cksum[/etc/passwd],#2)

This trigger fires if /etc/passwd has been changed. The expression is true when the previous /etc/passwd checksum differs from the most recent one. Similar expressions could be useful to monitor changes in important files, such as /etc/passwd, /etc/inetd.conf, /kernel, etc.
Data item: vfs.file.cksum

Detect changes in DNS query

last(/Zabbix server/net.dns.record[192.0.2.0,{$WEBSITE_NAME},{$DNS_RESOURCE_RECORD_TYPE},2,1])<>"{$WEBSITE_NAME}           {$DNS_RESOURCE_RECORD_TYPE}       0 mail.{$WEBSITE_NAME}"

Notice the quotes around the second operand.

This trigger fires if the query result deviates from what it normally returns:

example.com           MX       0 mail.example.com

Data item: net.dns.record[192.0.2.0,{$WEBSITE_NAME},{$DNS_RESOURCE_RECORD_TYPE},2,1], with macros defined as:

{$WEBSITE_NAME} = example.com
       {$DNS_RESOURCE_RECORD_TYPE} = MX

Detect non-matching software on different hosts

last(/host/vfs.file.contents[/etc/os-release])<>last(/host2/vfs.file.contents[/etc/os-release])

This trigger fires if the Ubuntu version is different on different hosts. Note how operands here are functions that return strings.
Data item: vfs.file.contents

Detect unsynchronized clocks

fuzzytime(/host/system.localtime,10s)=0

The trigger fires if the client local time and Zabbix server time differs by more than 10 seconds.
Data item: system.localtime

Note that system.localtime must be configured as a passive check for Zabbix agent; on Zabbix agent 2 it may be configured as an active check.

Detect outdated agent

find(/host/agent.version,,"like","beta")=1

This trigger fires if Zabbix agent has beta version. Zabbix agent needs to be upgraded.
Data item: agent.version

Detect flapping interface

changecount(/host/vfs.file.contents["/sys/class/net/eth0/operstate"],1h)>5 

This trigger fires if the operational state (up/down/unknown) of eth0 has changed more than 5 times in an hour.
Data item: vfs.file.contents

Detect high incoming traffic

min(/host/net.if.in[eth0,bytes],5m)>100K

This trigger fires if the number of received bytes on eth0 within the last five minutes was always over 100 kilobytes. Someone is probably downloading a large file.
Data item: net.if.in[eth0,bytes]

Detect low disk space

Problem expression:

max(/host/vfs.fs.size[/,free],5m)<10G

The trigger fires if free disk space is consistently (5 minutes) below 10 GB.

Recovery expression:

min(/host/vfs.fs.size[/,free],10m)>40G

The problem gets resolved when free disk space is consistently (10 minutes) above 40 GB.
Data item: vfs.fs.size

Detect low disk space (adaptive threshold)

last(/template/hrStorageFree[{#SNMPVALUE}])<last(/template/hrStorageSize[{#SNMPVALUE}])*0.1

The trigger fires if free storage (in allocation units) drops below 10 percent. Notice the value of another item being used to get an adaptive trigger threshold, applicable to discovered storage of various size.

Detect high CPU load

last(/host/system.cpu.load[all,avg1])>5

The trigger fires when the average processor load has been above 5 for one minute.

Variations:

min(/host/system.cpu.load[all,avg1],5m)>2 and time()<060000

min(/host/system.cpu.load[all,avg1],5m)>2 and not (dayofweek()=7 and time()>230000) and not (dayofweek()=1 and time()<010000)

Such triggers analyze 5 minutes of data and trigger only if CPU load is never under 2. Additionally, these triggers fire:

• only at night time (00:00-06:00)
• at any time except for 2 hours on week change (Sunday, 23:00 - Monday, 01:00)

(last(/host/system.cpu.load[all,avg1])>5) + (last(/host2/system.cpu.load[all,avg1])>5) + (last(/host3/system.cpu.load[all,avg1])>5)>=2

This trigger fires if the processor load is too high on at least two of the three hosts. Data item: system.cpu.load

Compare CPU loads

avg(/Zabbix server/system.cpu.load,1h)/avg(/Zabbix server/system.cpu.load,1h:now-1d)>2

The trigger fires if the average load today tops the average load of the same hour yesterday (using time shift as now-1d) more than two times.
Data item: system.cpu.load

Compare long-term CPU loads

trendavg(/Exchange/system.cpu.load,1M:now/M)>1.1*trendavg(/Exchange/system.cpu.load,1M:now/M-1M)

This trigger fires if the load on the Exchange server increased by more than 10% last month

You may also use the Event name field in trigger configuration to build a meaningful alert message, for example to receive something like

"Load of Exchange server increased by 24% in July (0.69) comparing to June (0.56)"

the event name must be defined as:

Load of {HOST.HOST} server increased by {{?100*trendavg(//system.cpu.load,1M:now/M)/trendavg(//system.cpu.load,1M:now/M-1M)}.fmtnum(0)}% in {{TIME}.fmttime(%B,-1M)} ({{?trendavg(//system.cpu.load,1M:now/M)}.fmtnum(2)}) comparing to {{TIME}.fmttime(%B,-2M)} ({{?trendavg(//system.cpu.load,1M:now/M-1M)}.fmtnum(2)})

It is also useful to allow manual closing in trigger configuration for this kind of problem.

Detect string containing special characters

last(/host/vfs.file.contents[/tmp/hello])={$HELLO_MACRO}

The trigger fires if /tmp/hello content is equal to the string defined in {$HELLO_MACRO}:

{$HELLO_MACRO} = \" //hello ?\"

Alternatively, you can compare to the string directly:

last(/Zabbix server/vfs.file.contents[/tmp/hello])="\\\" //hello ?\\\""

Notice how the special characters (\\ and ") characters are escaped when the string gets compared directly.

Data item: vfs.file.contents