An idea to update Zabbix Agent automatically via zabbix

[r0k5t4r]

Hi All,

Im running Zabbix 1.8.3 and I monitor a lot of Unix and Linux hosts (200+). I recently upgraded to 1.8.3 and updating the agent on so many machines is pointless. We currently don't have a central package management for Unix/Linux in place. I thought this might be a good idea to have this done by Zabbix itself. So I created a new Item that tells me that Zabbix agent verison running on a host:



And the following trigger:



This seems to be working fine. In the Dashboard I can now see the hosts under severity information:



So the trigger is working as expected. Last thing we need is an action:



Here is the problem the action doesn't seem to work. This is the last missing bit. I have a central action to send an notification email for severity=information and severity=average. Average severity seems to work. But there is no notification email send for information severity.I don't know if it has to do with the new item? If I could solve this I only need to write a small shell script that is triggered by the Zabbix server. The script could be located on a nfs share and mounted via autofs.

Please help. I think this function would be very useful for others as well.

Regards,
r0k5t4r

[jthakrar]

First, this is a nice idea !

If I look at the trigger, it has been defined with severity information.
However, in looking at the action, the condition for the action does not include this severity (and you seem to be using the trigger value). Try using trigger severity with value as Informational and see if it works.

Please do post your findings once you find the solution......

Jayesh

[r0k5t4r]

Hi,

sorry for the very very late reply, but I've been to busy to work on this. Today I tried your suggested change but unfortunately this doesn't solve the problem either.

I did some research and created a simple action to restart vsftpd once it is down. The host is running ubuntu linux and i made the following sudoers entry:

Cmnd_Alias      ZABBIX = /etc/init.d/vsftpd restart
zabbix ALL = (ALL) NOPASSWD: ZABBIX

This is working just fine.

I configured my update action exactly the same like the vsftpd restart action plus a notification:



This somehow doesn't work. I have no idea why. The Trigger works fine since I get a notification. But the action is not taking place.

Regards,
Oliver

[r0k5t4r]

Ok, the problem is found. It's because the machines are managed by a proxy and remote commands are still not working through proxy. Only chance is to turn proxy in a full zabbix server.

[EnigmA-X]

Are you aware that Zabbix offers full compatibility between servers/agents/proxies on the same minor-versions. I.e., all server/agent/proxy versions of 1.8x will be compatible.

For my situation, I'm running 1.8.1, 1.8.2 and 1.8.2 servers/agents without any problem.

For the server (and frontend) it's a good thing to keep this up-to-date as suited. For agents, I don't see a real need because functionality doesn't differ very much. At least, I did not encounter any bugs/problems so far to start upgrading these

[nelsonab]

Servers are backwards compatible, agents are not. Thus you can run a 1.1 agent with a 1.8 server.

[vlam]

have anyone looked into this again for the newer versions.

I have a combination of Windows and Unix servers spread over 10 Proxies and 500 agents and they all vary between the version 2 variants and I need to upgrade the entire environment to version 2.4

Any info if this is possible would be greatly helpful

Thanks

[tiarnaigh]

Howdy folks.
This is actually something I ran into myself also over the past few months/years (i.e. the pains of trying to keep Zabbix agents up to reasonably latest version). One of the coolest/handiest features I found in Zabbix for doing this was via the remote agent calls, which from version 2.x onwards work pretty well (Still not quite as powerful as say SaltStack, but not far off!). Granted, you'll need to allow remote calls on all agents, but this is something we do in our environment and it works very well so far.

Anyways, long story short. We build all Zabbix agents from source, and then simply update to latest version each time by pulling down latest source and building accordingly (If you've done rpm based installs, then the update process should be even easier - i.e. rpm -Uvh zabbix-latest.rpm via remote call). Attached is the custom script I used last time around for going from zabbix-2.4.0 up to zabbix-2.4.3:


Basically, all this does is update the default Timeout on the remote agent initially from default 3 secs to 30 seconds (This is something you should only have to do once).
It then restarts the Zabbix agent for this to take affect.
Once Zabbix is restarted, it then runs configure & make install against the new version (2.4.3 in this case), followed by an additional agent restart. Finally, it validates the
version which is currently running and then removes the install files


To run the above script against all hosts listed in Zabbix should be as simple as the below (Dump the resulst of this out to a text file say):

zabbix=# select distinct(ip) from interface where ip <> '127.0.0.1' order by ip;
ip
---------------
10.10.1.208
10.10.1.209
10.10.1.24
..............
...............


Finally, loop all IPs through the original script as per below:

[root@zbx2dal0 ~]# cat outStanding.txt | while read i; do ./upgradeZabbixAgent.sh $i; done
############################
Current Zabbix agent version
2.4.2
############################
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 51.0M 100 51.0M 0 0 82.6M 0 --:--:-- --:--:-- --:--:-- 87.7M



############################
Restarting Zabbix agent. Please standby....................
1
############################
############################
Configuring new Zabbix agent
############################




This may not work for everyone I guess, depending on their Zabbix layout/configurations, but for me at least it works pretty well so far.

Hope you find this useful,
M

[tiarnaigh]

Just ran zabbix-2.4.4 upgrade (Was previously running zabbix-2.4.2) on all remote agents this morning (47 agents took approximately 10 mins) using the script I outlined above:

[root@zbx2dal0 php]# runuser -l postgres -c 'psql -d zabbix -c "select distinct(ip) from interface order by ip;"'
      ip
---------------
 10.149.36.200
 10.149.36.208
 10.149.36.216
 ..................
 ..................
 ..................

Save above results out to text file (Say hosts.txt), and then run upgrade script below for each as follows:

cat hosts.txt | while read i; do sh upgradeZabbixAgent2.4.4.sh $i;done

#!/bin/bash

# This script is to be run on each remote agent. Basically, it pulls down the latest zabbix tar ball from a remote server, extracts it locally
# and then re-builds the new version from source via "configure" & "make install". Finally, it restarts the remote zabbix agent and removes the original
# install files

IP=$1

echo "############################"
echo "Current Zabbix agent version"
/sys_apps_01/zabbix/bin/zabbix_get -s $IP -k agent.version;
echo "############################"
/sys_apps_01/zabbix/bin/zabbix_get -s $IP -k system.run["curl -k https://10.149.1.10/zabbix-2.4.4.tar -o /tmp/zabbix-2.4.4.tar","wait"]
/sys_apps_01/zabbix/bin/zabbix_get -s $IP -k system.run["cd /tmp; tar -xf zabbix-2.4.4.tar","wait"]
/sys_apps_01/zabbix/bin/zabbix_get -s $IP -k system.run["sed -i 's/ac_default_prefix=\/usr\/local/ac_default_prefix=\/sys_apps_01\/zabbix/g' /tmp/zabbix-2.4.4/configure","wait"];
echo "############################"
echo "Configuring new Zabbix agent"
echo "############################"
/sys_apps_01/zabbix/bin/zabbix_get -s $IP -k system.run["PATH=$PATH:/opt/java/jdk1.7.0_21/bin; cd /tmp/zabbix-2.4.4; ./configure --enable-agent --with-net-snmp --with-libcurl --with-ssh2 --enable-java --enable-ipv6","wait"];
echo "############################"
echo "Running make install on new Zabbix agent"
echo "############################"
/sys_apps_01/zabbix/bin/zabbix_get -s $IP -k system.run["PATH=$PATH:/opt/java/jdk1.7.0_21/bin; cd /tmp/zabbix-2.4.4; make install","wait"];
echo "############################"
echo "Restarting Zabbix agent. Please standby...................."
/sys_apps_01/zabbix/bin/zabbix_get -s $IP -k system.run["/etc/init.d/zabbix_agentd restart","nowait"];
echo "############################"
sleep 20
echo "############################"
echo "New Zabbix agent version"
/sys_apps_01/zabbix/bin/zabbix_get -s $IP -k agent.version;
echo "############################"
/sys_apps_01/zabbix/bin/zabbix_get -s $IP -k system.run["rm -rf /tmp/zabbix-2.4.4*","wait"]

Finally, validate as follows:

[root@zabbix01 ~]# cat hosts.txt | while read i; do /sys_apps_01/zabbix/bin/zabbix_get -s $i -k agent.version; done
2.4.4
2.4.4
2.4.4
.....
.....

Piece of pie!