1 Trigger examples

This page is a collection of trigger examples.

Examples are sorted by use case:

• Detect unavailable host
• Detect unavailable proxy
• Detect unavailable SMTP cluster
• Detect unreachable host
• Detect unexpected reboot
• Detect changes in important files
• Detect changes in DNS query
• Detect non-matching software on different hosts
• Detect unsynchronized clocks
• Detect outdated agent
• Detect flapping interface
• Detect high incoming traffic
• Detect low disk space (with hysteresis)
• Detect low disk space (adaptive threshold)
• Detect high CPU load
• Compare CPU loads
• Compare long-term CPU loads
• Detect string containing special characters

Detect unavailable host

max(/host/zabbix[host,agent,available],5m)=0

Zabbix agent on the host has been unavailable for 5 minutes.
Data item: zabbix[host,agent,available]

Alternative method:

nodata(/host/agent.ping,5m)=1

No data has been received from Zabbix agent for 5 minutes.
Data item: agent.ping

Detect unavailable proxy

fuzzytime(/host/zabbix[proxy,{$PROXY_NAME},lastaccess],1m)=0

Zabbix proxy data lags behind Zabbix server time by 1 minute.
Data item: zabbix[proxy,{$PROXY_NAME},lastaccess]

Detect unavailable SMTP cluster

last(/smtp1.example.com/net.tcp.service[smtp])=0 and last(/smtp2.example.com/net.tcp.service[smtp])=0

The expression is true when both SMTP servers are down.
Data item: net.tcp.service

Detect unreachable host

count(/host/icmpping,30m,,"0")>5

The expression is true if the host is unreachable by ping more than 5 times in the last 30 minutes.
Data item: icmpping

Detect unexpected reboot

change(/host/system.uptime)<0

A negative change in the system uptime value indicates a reboot.
Data item: system.uptime

Detect changes in important files

last(/host/vfs.file.cksum[/etc/passwd],#1)<>last(/host/vfs.file.cksum[/etc/passwd],#2)

/etc/passwd has been changed. The expression is true when the previous /etc/passwd checksum differs from the most recent one. Similar expressions could be useful to monitor changes in important files, such as /etc/passwd, /etc/inetd.conf, /kernel, etc.
Data item: vfs.file.cksum

Detect changes in DNS query

last(/Zabbix server/net.dns.record[192.0.2.0,{$WEBSITE_NAME},{$DNS_RESOURCE_RECORD_TYPE},2,1])<>"{$WEBSITE_NAME}           {$DNS_RESOURCE_RECORD_TYPE}       0 mail.{$WEBSITE_NAME}"

Notice the quotes around the second operand.

This trigger will fire if the query result deviates from what it normally returns:

example.com           MX       0 mail.example.com

Data item: net.dns.record[192.0.2.0,{$WEBSITE_NAME},{$DNS_RESOURCE_RECORD_TYPE},2,1], with macros defined as:

{$WEBSITE_NAME} = example.com
       {$DNS_RESOURCE_RECORD_TYPE} = MX

Detect non-matching software on different hosts

last(/host/vfs.file.contents[/etc/os-release])<>last(/host2/vfs.file.contents[/etc/os-release])

Create an alert if Ubuntu version is different on different hosts. Note how operands here are functions that return strings.
Data item: vfs.file.contents

Detect unsynchronized clocks

fuzzytime(/host/system.localtime,10s)=0

The trigger will fire if client local time and Zabbix server time differs by more than 10 seconds.
Data item: system.localtime

Note that system.localtime must be configured as a passive check for Zabbix agent; on Zabbix agent 2 it may be configured as an active check.

Detect outdated agent

find(/host/agent.version,,"like","beta")=1

The expression is true if Zabbix agent has beta version. Zabbix agent needs to be upgraded.
Data item: agent.version

Detect flapping interface

changecount(/host/vfs.file.contents["/sys/class/net/eth0/operstate"],1h)>5 

The operational state (up/down/unknown) of eth0 has changed more than 5 times in an hour.
Data item: vfs.file.contents

Detect high incoming traffic

min(/host/net.if.in[eth0,bytes],5m)>100K

The expression is true when the number of received bytes on eth0 within the last five minutes was always over 100 kilobytes. Someone is probably downloading a large file.
Data item: net.if.in[eth0,bytes]

Detect low disk space

Problem expression:

max(/host/vfs.fs.size[/,free],5m)<10G

The trigger will fire if free disk space is consistently (5 minutes) below 10 GB.

Recovery expression:

min(/host/vfs.fs.size[/,free],10m)>40G

The problem gets resolved when free disk space is consistently (10 minutes) above 40 GB.
Data item: vfs.fs.size

Detect low disk space (adaptive threshold)

last(/template/hrStorageFree[{#SNMPVALUE}])<last(/template/hrStorageSize[{#SNMPVALUE}])*0.1

The trigger will fire if free storage (in allocation units) drops below 10 percent. Notice the value of another item being used to get an adaptive trigger threshold, applicable to discovered storage of various size.

Detect high CPU load

last(/host/system.cpu.load[all,avg1])>5

The trigger will fire when average processor load has been above 5 for one minute.

Variations:

min(/host/system.cpu.load[all,avg1],5m)>2 and time()<060000

min(/host/system.cpu.load[all,avg1],5m)>2 and not (dayofweek()=7 and time()>230000) and not (dayofweek()=1 and time()<010000)

Such triggers will analyze 5 minutes of data and trigger only if CPU load was never under 2. Additionally, these triggers will fire:

• only at night time (00:00-06:00)
• at any time except for 2 hours on week change (Sunday, 23:00 - Monday, 01:00)

(last(/host/system.cpu.load[all,avg1])>5) + (last(/host2/system.cpu.load[all,avg1])>5) + (last(/host3/system.cpu.load[all,avg1])>5)>=2

The processor load is too high on at least two of the three hosts. Data item: system.cpu.load

Compare CPU loads

avg(/Zabbix server/system.cpu.load,1h)/avg(/Zabbix server/system.cpu.load,1h:now-1d)>2

The trigger will fire if the average load today tops the average load of the same hour yesterday (using time shift as now-1d) more than two times.
Data item: system.cpu.load

Compare long-term CPU loads

trendavg(/Exchange/system.cpu.load,1M:now/M)>1.1*trendavg(/Exchange/system.cpu.load,1M:now/M-1M)

Load on the Exchange server increased by more than 10% last month

You may also use the Event name field in trigger configuration to build a meaningful alert message, for example to receive something like

"Load of Exchange server increased by 24% in July (0.69) comparing to June (0.56)"

the event name must be defined as:

Load of {HOST.HOST} server increased by {{?100*trendavg(//system.cpu.load,1M:now/M)/trendavg(//system.cpu.load,1M:now/M-1M)}.fmtnum(0)}% in {{TIME}.fmttime(%B,-1M)} ({{?trendavg(//system.cpu.load,1M:now/M)}.fmtnum(2)}) comparing to {{TIME}.fmttime(%B,-2M)} ({{?trendavg(//system.cpu.load,1M:now/M-1M)}.fmtnum(2)})

It is also useful to allow manual closing in trigger configuration for this kind of problem.

Detect string containing special characters

last(/host/vfs.file.contents[/tmp/hello])={$HELLO_MACRO}

The trigger will fire if /tmp/hello content is equal to the string defined in {$HELLO_MACRO}:

{$HELLO_MACRO} = \" //hello ?\"

Alternatively, you can compare to the string directly:

last(/Zabbix server/vfs.file.contents[/tmp/hello])="\\\" //hello ?\\\""

Notice how the special characters (\\ and ") characters are escaped when the string gets compared directly.

Data item: vfs.file.contents