Including more information in alerts

[Quinn Comendant]

Is there a way to include more information in the alerts that zabbix sends?

Example 1: In an alert for "Processor load is too high ..." it would be useful to actually include the current processor load value. It would be more useful to output a page from top in the message to learn what caused the problem in that moment. Is it possible to include the output from an arbitrary script?

Example 2: In an alert for "/etc/passwd has been changed ..." it would be useful to see a diff of the before/after changes to know *what* changed. Again, this could be scripted by running and including the output from an external script.

Example 3: Possible to send regular (i.e. daily) alerts with current values? For example, month-to-date outgoing data usage at the port?

Cheers,
Quinn

[Alexei]

Example 1. See ZABBIX documentation to see how this can be done.

[niekie]

Instead of using a direct mail action, you could also define your own script, and have that one email you. That way you have full control over what you include in that email. With a bit of effort you can also get info from the DB using the parameters available to you, and even include graphs.
The script I use reports the trigger that fired, including a graph of the item in the last hour, and a graph for the last day and last week as well.

[g0rdy]

Can you share a sample Nieckie ?

[niekie]

An example trigger script is here:

http://www.qwertyboy.org/files/trigger.sh

You will have to set up a media type with this script, define a user who has that media, and then set up an action with the following message in the "operations" field:

Host:{HOSTNAME}
IP:{IPADDRESS}
Time:{TIME}
Key:{TRIGGER.KEY}
Name:{TRIGGER.NAME}
Stat:{STATUS}
Sev:{TRIGGER.SEVERITY}
Val:{ITEM.LASTVALUE}

The script will parse this "message body", and will generate the actual mail message.

Niek.

[bbrendon]

niekie- wow. Looks cool. i'll have to dig into your script a little deeper. thanks for sharing.

[jvella]

Permission error

I downloaded the trigger.sh script yesterday and installed it. It works to the extent that an email is being generated with info extract by the script. It also attached png files but the png files just have the error message

ERROR: No Permissions

Any clues?

[niekie]

The message "ERROR: No permissions" is likely caused by the security used within Zabbix. The script uses `wget` to retrieve the graphs, and for that to work the script tries to trick Zabbix into believing the `wget` came from a logged-in session. Zabbix uses a session table in its database, and validates HTTP GET requests with a cookie that has to exist in that session table. So before the script actually does the `wget` it inserts a self-defined session cookie in the session table, and subsequently does a `wget` using that same cookie.

Having said all that, looking at my script you downloaded it appears I have mistakenly left a test user and password hardcoded in the command that does the "insert" of the session cookie (line 88). That would explain the script not being able to do the insert, and hence there is no valid session cookie in the table, so the `wget` will fail.

Please change line #88 to have it look like line #99. I have updated the script on my web site to reflect this.

Niek.

[niekie]

The message "ERROR: No permissions" is likely caused by the security used within Zabbix. The script uses `wget` to retrieve the graphs, and for that to work the script tries to trick Zabbix into believing the `wget` came from a logged-in session. Zabbix uses a session table in its database, and validates HTTP GET requests with a cookie that has to exist in that session table. So before the script actually does the `wget` it inserts a self-defined session cookie in the session table, and subsequently does a `wget` using that same cookie.

Having said all that, looking at my script you downloaded it appears I have mistakenly left a test user and password hardcoded in the command that does the "insert" of the session cookie (line 88). That would explain the script not being able to do the insert, and hence there is no valid session cookie in the table, so the `wget` will fail.

Please change line #88 to have it look like line #99. I have updated the script on my web site to reflect this.

Niek.

[jvella]

Thanks niekie but I had already found that little problem. However, the issue was relating to the insertion of the cookie. The line that inserts the cookie into the sessions table requires another field value as shown in red.

echo "insert into sessions values (\"${COOKIE}\",\"1\",\"`date '+%s'`\",\"0\");" \

One step closer.

The trigger.sh script now works if run from the command line but I can't get the script to execute from zabbix.

-I have setup the Media Type as type "script" with the appropriate script name.

- I have setup a user with that media type.

- I have setup an action with an operation as follows:
operation type: send Message
Send Message to the user with the appropriate media type
Subject: {Trigger.name}: {status}
Message: Host:{HOSTNAME}
IP:{IPADDRESS}
Time:{TIME}
Key:{TRIGGER.KEY}
Name:{TRIGGER.NAME}
Stat:{STATUS}
Sev:{TRIGGER.SEVERITY}
Val:{ITEM.LASTVALUE}

Any further clues?

Thanks

[niekie]

Keep in mind that that script runs in the context of the Zabbix server process (usually that means it runs as user "zabbix"). If you are logged in as root, try "su - zabbix" and run the script again - it may give you messages that points to the issue (it sounds like a file permission or wrong PATH env var to me).

Niek.

[jvella]

Success

Again thanks for the clues. In the end the problem was that zabbix assumes that the script is in /home/zabbix/bin/. I had placed my script in another folder and hence it was not being found.

Script now works as specified. Very handy.

Thanks for the help.

[bbrendon]

Did you check in zabbix_server.conf?

AlertScriptsPath=/etc/zabbix/alert.d/

[gospodin.horoshiy]

An example trigger script is here:

http://www.qwertyboy.org/files/trigger.sh

You will have to set up a media type with this script, define a user who has that media, and then set up an action with the following message in the "operations" field:

Host:{HOSTNAME}
IP:{IPADDRESS}
Time:{TIME}
Key:{TRIGGER.KEY}
Name:{TRIGGER.NAME}
Stat:{STATUS}
Sev:{TRIGGER.SEVERITY}
Val:{ITEM.LASTVALUE}

The script will parse this "message body", and will generate the actual mail message.

Niek.

This is great, thnk you !!

Unfortunately for me, script doesn't work out-of-the-box. I receive pictures attached which say "itemid is not an integer".

I think this is because key in my trigger's key is perf_counter["\4\1406"] so contains " and \ escape characters
In trigger.sh there are lines:



HOSTID=`echo "select hostid from hosts where host = \"${D_HOST}\";" \
| mysql -u${MY_USER} -p${MY_PASS} zabbix \
| tail -1`
ITEMID=`echo "select itemid from items where hostid = \"${HOSTID}\" and key_ = \"${D_KEY}\";" \
| mysql -u${MY_USER} -p${MY_PASS} zabbix \
| tail -1`



because D_KEY = perf_counter["\4\1406"] - SQL query doesn't work and returns ITEMID = NULL or something like that...

Is there any way to work around it?

[bbrendon]

Is there any way to work around it?

Yes, I have the same problem. Since setting it up, I haven't gone back to squash that bug.

I'll be watching for the proposed fix.