Using Centerim for Instant Messaging notifications (and more)

[qix]

A while back I was looking for a nice way to use IM messages as notifications for Zabbix events.

I found that GTalk isn't supported with the current Jabber implementation in Zabbix, so I went looking for an alternative. Most scripts I found didn't work or were to dependent on other scripting languages I didn't want on my system.

I figured, if Pidgin can do it, there must be some nice way to do it from the command line also

Then I found Centerim which is something of a Pidgin for the command line. It supports Jabber (and GTalk), MSN, ICQ, Gadu Gadu and some other IM protocols.

If I could get this to work, It would mean a central way of doing IM notifications for almost all protocols.

You can send messages from the command line by piping text into a centerim command. This will queue the message for delivery. If centerim is running, it will be delivered automatically, else it will sit in queue until Centerim gets started.

I currently have done the following:

• I set up a new gmail/hotmail/whatever account for my monitoring server to use as a source IM account.
• Installed Centerim and screen on my Zabbix server/
• Logged in as the Zabbix user on my server and setup the configuration for my source accounts in Centerim
• I run the following command in my bootup script to make centerim run in the background as the zabbix user:

  su - zabbix -c "screen -d -m centerim"

• I created a IM script for sending IM messages in the alert scripts directory on the zabbix server (see below).
• Defined a new media type in Zabbix called IM.
• added media for my user account in the form of 'jab:[email protected]' (see centerim documentation for other protocol abbreviations) to start using my Gtalk account for notifications.

And it's working, although not yet as tidy as I like it to be. The version of Centerim that is shipped with Debian Lenny has broken MSN support, so I'll need to do a manual install to use that.

Also the scripts need to be made a bit more robust.
It would be handy to find out a way to restart centerim of something goes wrong. Maybe Zabbix can do this with remote commands...

Anyway, here is the script I'm using currently:

#!/bin/bash
#
# Use centerim as IM gateway for Zabbix Notifications.
# Center IM must be running in the background under the same user Zabbix is running as.
# Please start Centerim as user zabbix manually to setup the centerim configuration prior
# to running it in the background.
#
# You can use su with screen to run it in the background as follows:
#
# su - zabbix -c "screen -d -m centerim"
#
# Place this in your startup scripts to get Centerim running at boottime.
# Make sure Centerim is fully configured and running.
#
# The script expects to find the recipient in the following way:
#
# <protocol>:<IMuser>
#
# See the Centerim documentation for the valid protocols.
# For a GTalk user this would be specified in zabbix as:
#
# jab:[email protected]
#

CENTERIM_PATH=/usr/bin
CENTERIM_BASE=/home/zabbix/.centerim
PROTO=`echo $1 | cut -d":" -f1`
RCPT=`echo $1 | cut -d":" -f2`
SUBJECT=$2
MSG=$3

echo "$SUBJECT" | $CENTERIM_PATH/centerim -b $CENTERIM_BASE -s msg -t $RCPT -p $PROTO > /dev/null
echo "$MSG" | $CENTERIM_PATH/centerim -b $CENTERIM_BASE -s msg -t $RCPT -p $PROTO

It will first IM you the subject and then the body of the event notification.

I'm also using the attached template to make sure CenterIM is running on my server.

As an added feature, CenterIM allows scripting of responses to incoming IM messages. Since it keeps the source account online, you can use this to send queries to the zabbix server.

I made the following 'external' script for centerim (please see centerim documentation for more info):

%action         Answering machine
event           msg
proto           all
status	 all
options         stdout

%exec
#!/bin/sh
if [ $CONTACT_NICK != [email protected] ];
then


cat <<EOF
Hello,

This is an automated system response.
Please contact [email protected] for more information.

--
Your friendly Zabbix Daemon
EOF

else
exit 0
fi

%action         Remote control
event           msg
proto           all
status          all
options         stdin stdout

%exec
#!/bin/sh
if [ $CONTACT_NICK = [email protected] ]; then 
echo " "
case `cat` in
    df) df -h;;
    uptime) uptime;;
    zbxlog) tail -n 20 /tmp/zabbix_server.log;;
    ifstat) ifstat 1 5;;
    iostat) iostat;;
    vmstat) vmstat;;
    cookie) /usr/games/fortune;;
    ntpq)   ntpq -pn;;
    zbxstatus) if [ `ps ax | grep zabbix_server | grep -v grep | wc -l` -ge 1 ]; then echo Zabbix is ALIVE; else echo Zabbix is DEAD;fi;;
    *)      echo "I'm sorry Dave, I'm afraid I can't do that.";;
esac

else

exit 0

fi

Replace the [email protected] for your own admin account.
It sends an automated response to anyone not being the admin user.
The admin user can ask for the current state of some system/zabbix related commands. I think it should be obvious what they do for most of you.

I threw the cookie command in just for fun.The movie freaks should recognize the 'command not found' reply I used

I would really like to extend the commands with things like 'zbxevents' to show a list of the last occured events or 'zbxtriggers' to show the currently active triggers. I was looking into zabcon for this, but I haven't figured it out yet.

Any help or ideas on further extensions are welcome!

[qix]

Oh, If you have problems getting Gtalk to work, check out Centerim documentation. It requires some special parameters in the config file.

See here: http://www.centerim.org/index.php/Documentation.

[untergeek]

I got centerim to work on RHEL5 and did some testing. I found that it queues notifications sent in this manner. The queue doesn't seem to be flushed for somewhere between 30 seconds and 1 minute.

With the volume of notification I expect, and the time-sensitive nature of server notifications, I'm a bit disappointed in the time delay. It's not a deal breaker, but it's really a bummer to get a prowl notification to my phone so much sooner than an IM. I mean, the "I" in IM is supposed to mean instant…

For now, I'm satisfied with the perl -> GTalk plugin I have as it is instantaneous. If others at my work wish to use Yahoo, AIM or MSN then I will reconsider using centerim at that point. Until then, it's a tool in my toolbox waiting for use.

[qix]

For me, that kind of time delay is not a problem, but I can see that it will not be sufficient in some use cases. I also use Prowl on the side for the high and disaster level triggers. Everything lower than that level is IM/Email only.

I also really like that I can interactively communicate with the zabbix server. I hope Zabcon is going to get somewhat better so that I can query current trigger status directly via IM. I'm also curious if it would be possible to directly acknowledge an event from my IM client, that would certainly be time-efficient.

That I can now use a lot of IM protocols in one go is very neat as well.

That kind of added 'bonus features' make it very interesting IMHO

[untergeek]

Interesting indeed. My own company would never permit us to have external access to potentially run commands in our DMZ through this "gateway," if you will. I do see the potential, however.

As far as acknowledgement goes, I do like the URL being in the alert, as in the other thread (I included my example). I can, with a single click, open a browser window on my computer (or my iPhone) and acknowledge an alert. It's pretty remarkable. I wonder if there were a way to put pre-determined text into the field in advance, like an extra &alert_text="blah blah blah" at the end of the URL.