scheduling enable/disable monitoring on some hosts

[LEM]

Well, ZABBIX 1.1.x does not have its own scheduler (yet), anyway, sometimes we want to automagically disable monitoring on some hosts (say for scheduled maintenances windows) and re-enable this hosts at another date/time.

Thank's to cron and this little simple and stupid perl script (see at the end of this post), this can be quickly set up:

• grab the script and place it somewhere (/etc/zabbix/bin/ ?),
• create two links (hardlinks?) pointing at this script: zabbix_disable_hostlist and zabbix_enable_hostlist
• edit the script and place the correct-to-your-installation databasename, username, password to access the ZABBIX database
• use your-prefered-text-editor to create an host list
• use crontab to schedule auto-activation/deactivation

Here what you crontab should look like if you want to automatically disable the /etc/zabbix/sap-servers-list at 08:00 each monday, and reactivate monitoring on these hosts at 20:00 the same day:

0 8 * * 0 /etc/zabbix/bin/zabbix_disable_host /etc/zabbix/sap-servers-list
0 20 * * 0 /etc/zabbix/bin/zabbix_enable_host /etc/zabbix/sap-servers-list

Here's the script: http://sholari.free.fr/zabbix/zabbix_switchhost. I use it with 1.0 release, but it should be ok for 1.1.x release too.

Hope this'll helps some of us.

Btw, what about a 'scheduler' included in ZABBIX 1.xxx ? The kind of scheduler where it could be possible to select: hosts, hostgroups, items, actions... and decide to activate/deactivate them based on date/time and/or timeframe?

[primos]

Yes this would be nice, I have schedular in my added db table with gui in zabbix for this purpose. Works the same , but I can configure it from web interface.

lp, p

[LEM]

I have schedular in my added db table with gui in zabbix for this purpose. Works the same , but I can configure it from web interface.

Share the Code, Luke ! :-)

[cybaspace]

Hey that's cool! But will it be possible to do the same for just the triggers? We want to measure some machines during the night but we don't want to see the active triggers on some machines.

I know that I can put that in the trigger expression but it would be a lot easier this way.

[LEM]

Adapting the given script to deal with triggers would be easy using triggerIDs (and flaging the triggers status)... but well, it's more a hack (which can be of use), to propose/illustrate a proposed feature (ZABBIX internal scheduler).

[RobertS]

I found an other possibility to disable triggers during maintenance:

1. Monitor "status" on the selectes host/template
2. Create a trigger called "maintenance" and set the key to "({host:status.time(0)}>010000)&({host:status.time (0)}<030000)", set the severity to "information" only
3. Let all the other triggers of the host depend on this trigger

The "maintenance trigger" becomes true from 01:00am to 03:00am, so all the other triggers will not become true during ths period. You can shutdown the servers in this time or whatever you want without getting a message.

There is only on disadvantage: When a server or service does not come up in this period, the trigger will be set and displayed after 03:00am, but there will be no action/message.

Regards, Robert

[LEM]

Yes, you're right, playing with 'time' may helps not to alert (actions) during some predefined timeframes.

But, most of the time, we face the "hey sysadm guys, we're gonna make some change on this list of server this night... probably this will be done this weekend too... could you schedule monitoring disable/enable for us please"... and so, a simple scheduler should helps. That's why we ended with this little perl script to help us do the job...

[dcrandall]

Variation on maintenance window ideas.

I found an other possibility to disable triggers during maintenance:

1. Monitor "status" on the selectes host/template
2. Create a trigger called "maintenance" and set the key to "({host:status.time(0)}>010000)&({host:status.time (0)}<030000)", set the severity to "information" only
3. Let all the other triggers of the host depend on this trigger

I like this idea because just the triggers are silenced, which means that trends and graphs are not interupted. So if all that is happening is that somebody is bouncing apache, you still get uninterrupted history for unaffected items, and the machine doesn't fall off of any graphs it's on.

But, I like the perl script with hosts list because it's so easily modifiable - though I would probably change it to disable triggers instead of hosts for the reasons above.

This just leaves the issue of not receiving alerts from problems that occur during the maintenance window. As was pointed out when the maintenance window ends, no alerts will be sent for triggers that turned ON during that period.

I propose this modification for both the script and item based methods:

When the maintenance window is over, connect to the database and reset all of the hosts triggers to OFF. If the host is really down (or having some other problem that you'd like to get an alert for) then next iteration of the item checks would turn any problem triggers back to ON, which will generate an alert.

For the script method you'd just add this function in.

For the maintenance window item method, you could write a script to modify the trigger status in the database, and then create a trigger for the maintenance window item that runs the script when the window ends.

Yes, we are way off in hacksville with this, but ya gotta do what ya gotta do.

Great ideas all around!

[cperera]

I assume we're still in hacksville with scheduled outages

I was shooting ideas with a coworker and we decided on something (be it a perl/shell script or a PHP page) that let's you specify, the host, the trigger (optional), startime, and endtime. We're all LAMP hackers so that shouldn't be a problem.

I'm thinking out aloud here, so if you see a hole in my logic let me know. Each trigger which has a unique triggerid in the DB has a status of '0' when it's enabled and '1' when disabled. Simply changing this value in the DB will disable the trigger - I tried it manually and it works as expected. If an item changed to a status that should have triggered something, once you change the status of the trigger back to '0' it triggers as soon as the next update is received.

Now we can use our friendly old scheduler 'atd'. We'll have one job executed at starttime which sets the status of the trigger to '1', and a second job will be executed at endtime to set the status to '0'.

Comments?

[bbrendon]

cperera- I'm thinking along the same lines ...

A script for me would activate maintenance on groups of servers. Individual servers would be workable as well, but isn't a feature i'm looking for. Both would be nice

Option 1. Enabling maint mode would:
a. Find all triggers in a group or for the desired server
b. Save trigger status setting some where
c. Disable all triggers per #a above

Option 1. Disabling maint mode would:
a. Find all triggers in a group or for the desired server
a. Load all saved triggers from when maint was enabled
b. Update database with saved trigger settings, thus enabling them

Option 2:
-Have actions for every group and disable/enable actions to disable/enable maintenance mode.

[cperera]

b. Save trigger status setting some where

Ah yes, definitely need to do that.

[ulukay]

i've tried the status.time trigger workaround but that's not working
is there any other approach for the latest zabbix version? (1.4.4)

[rastaferraille]

Hi,

Could we do the same with days instead of hours ? a trigger on witch some others would depend testing days of week ?

Thans and sorry for my poor english

[exkg]

Hi,


This feature exist in ZABBIX 1.8 (1.7 beta).


[]s,
Luciano