Zabbix cluster (multiple locations)

[hmiddelhoek]

I'm trying to find out if it's possible with Zabbix to have multiple monitoring servers on multiple locations. In the manual I dicovered proxies and distributed monitoring, but the information provided in the manual didn't convince me of what I want to do.

We have servers spread over multiple locations across the country, one of these is the primary location and that's where our nagios installation is at the moment. But when this location had communication problems to the internet it can't send me an e-mail about it, or SMS with an e-mail->SMS gateway.

So, I want an approach where on every location (3) an Zabbix server is installed. Each server need to independently check ALL services and the results need to be collected. When 2 or more Zabbix servers report a service down the e-mail/SMS may be send out. This way we're always get reported in case of internet communication problems on one location, and we can't get false positives.

I'm looking for this kind of approach, it doesn't have to be exactly what I described, but I need to reach the same goal.

[nelsonab]

Short answer no.

Clustering like what you are looking for is somewhat tricky to do with the DB model of Zabbix. It's not impossible, just tricky and right now not possible.

A possible work around to your problem which is nowhere as complex is to get a SMS modem at each location and use that. The modem can either be an analog one (old school 56K connected to a POTS line, and yes it is possible to get one in most Colos) which dials the SMS service for the telco or a GSM/CDMA/other modem from a company like Multitech and use that to send SMS directly over the air. The advantage of both is you can still get messages if the internet goes down.

[hmiddelhoek]

Thanks for the quick answer.
SMS modem or something like that is a possibility we thought about, but it isn't a solution for (very exceptional) power failures. Other servers within the cluster could take care of that. That way it's close to perfect.

Well, I don't think other systems can do this, I also read about Zenoss and Nagios, with the latest it seems to be possible, but configuring and maintaining is far from user friendly and prone to errors.

Is there a way to limit the notifications? At the moment, with Nagios, we got dozens of SMS messages when a switch fails, on 2 cell's, at recovery the same happens. Is Zabbix able to set a limit of sending e-mails/SMS? After 10, stop sending for example? I think I'm awake after 10 messages and understand the message of Zabbix :-)
We want to monitor hosts (servers) and services (http/smtp/pop/imap/etc).

[ritontor]

Has this changed in the past six months? Because it seems pretty useless to have monitoring software that is prone to its underlying hardware failing.

[nelsonab]

Has this changed in the past six months? Because it seems pretty useless to have monitoring software that is prone to its underlying hardware failing.

The more important question is how often does Zabbix fail requiring the effort of clustering? Answer not very often.

There have been writeups over the years for setting Zabbix up to use Linux-HA, and more recently pacemaker. None of these are instant failover solutions and require anywhere from 15 seconds to one minute to fail over. If you MUST have failover work that quickly then you need to look at hardware from a company like Stratus.

This all comes down to a cost benefit analysis. Are you going to benefit from the additional cost and effort for clustering Zabbix? If your answer is yes then by all means go forth and put in the effort. If you need someone to come in and help set up your cluster send me a message and I can put you in touch with someone from Red Hat to discuss having a consultant come out and set up the cluster for you onsite.

[ritontor]

The more important question is how often does Zabbix fail requiring the effort of clustering? Answer not very often.

There have been writeups over the years for setting Zabbix up to use Linux-HA, and more recently pacemaker. None of these are instant failover solutions and require anywhere from 15 seconds to one minute to fail over. If you MUST have failover work that quickly then you need to look at hardware from a company like Stratus.

This all comes down to a cost benefit analysis. Are you going to benefit from the additional cost and effort for clustering Zabbix? If your answer is yes then by all means go forth and put in the effort. If you need someone to come in and help set up your cluster send me a message and I can put you in touch with someone from Red Hat to discuss having a consultant come out and set up the cluster for you onsite.

Yeah look, I suppose what I'm looking at isn't so much super HA, rather that we have a single VMware cluster that hosts our entire infrastructure, and obviously if i put the monitoring tools on that cluster, and we had the entire cluster fail, we aren't going to hear about it.

What I suppose most people do is they have an external monitoring service, just something hosted at a third party, and that way the odds of both of them going down are pretty slim. it'd just be very nice to have the ability to set up TWO external sites, and have the events only trigger when both monitoring setups agree that there is definitely an issue - i don't want to get 4am SMS alerts because the internet is getting twitchy.

up until now we've been looking at the online monitoring providers, like Site 24/7, etc, and they certainly cover off this sort of thing as a top line requirement, but they tend not to do so well when it comes to the deeper inspection of the servers, such as historical CPU usage data, etc., and for those who DO do it all, they're horrendously expensive.

[nelsonab]

Have a look at www.tribily.com, they're hosted zabbix