how to do web monitoring across a large farm?

[Crazy Marty]

I currently have some 30 web servers that need to pass various checks. By the end of the quarter, I will have some 50 more!

I (almost) have the hang of creating a web monitor for a single host, but it seems like this should appear in a template so that all the hosts that share the template "automagically" get the same web check.

I don't see any way of doing this currently, and it pains me greatly to think I might have to add them all individually.

Am I missing something here?

[Kees Jan Koster]

Dear Crazy Marty,

To answer that question we really need more information than you gave us. For example, are these 30-80 copies of the same site? Are all sites unique? How many clicks are in the click path? Are these all individual customers or is this output from a development department?

From what little information you gave, it sounds like you may need a specialised tool for this job and maybe feed the results to Zabbix, instead of letting Zabbix monitor all of these sites.

Perhaps it is an idea to look at jmeter? It allows you to record the click stream from a regular browser and saves the script as an XML file. You can edit that file with any editor and replay it using jmeter's scripts.

It has facilities to load variables from a file, so you can use it to store names and passwords in a separate file, or maybe hostnames if you have many identical sites on different hosts.

I'm sure there are hunderds of tools out there that do the same. I'm only suggesting jmeter because I am confident it can do what I suggested.

Kees Jan

[Crazy Marty]

Sorry for not giving enough info.

The hosts are identical: they are on the back end of (hardware) load balancers. Each has a public-facing web server on port 80, and several other services that use HTTP as their protocol on various other miscellaneous ports.

Mostly, all I need is to send a single GET request and match a single word in the response (but they'll be different requests and different match words for each service). I don't need any login sequence at all for any of the services.

I was thinking I could tie a few "web monitors" to a template, and have it automagically parameterize the hostname part of the URL, but I don't see how to do that.

Thanks for any other input!

[Kees Jan Koster]

Dear Crazy Marty,

Hmm. If the web sites can be monitored with a single GET request I would probably choose to monitor them using fetch/curl/wget and zabbix_sender. Something like:

fetch http://host/path | grep 'phrase I expect' | wc -l | zabbix_sender ...

Then schedule that in cron.

Maybe one of the Zabbix gurus knows a better solution.

Kees Jan

[Crazy Marty]

Hmmm. Except that I can't push data from the monitored machines to the Zabbix server (we have a high degree of compartmentalization using firewalls). And also, I like the data that appears to be collected from the "web monitor" stuff.

Actually, it occurs to me that I'd also like to be able to pull data from the equivalent of apache's "/status" (with "ExtendedStatus" enabled): thread/process counts, data rates, etc. Isn't anyone else interested in this sort of data?

[Crazy Marty]

web scenarios really want to be template members

If only a web scenario could be a member of a template, I'd be just ecstatic. As discussed above, I'd ideally add a web scenario to the template that I already have to measure perfomance items for the various applications. As things stand now, I'd have to create an application for each machine running that (web) server. I don't really get how to make good use of web scenarios as implemented today.

Alexei, have I completely missed the boat here? What am I missing?

[abederov]

Hello,

I am in the same boat here.

I have a few web servers here and they all run the same application. I'd love to check performance of 3-4 URLs on each machine and run a script on each box if one of these checks fails.

I see how I can configure standalone machine checks, but I cannot figure out how to use templates in this case. Oh, and I tried web.perf and unfortunately it does not have any configurable timeouts, so if it can connect, but it takes web server very long to respond, it does not work.

Alex

[noxis]

We have sorted this at our place with a perl script, both living on the server called by the agent and with an external script.

The script basically connects to localhost, gets the default page of the desired virtual host and times how long it takes (result is a number in seconds or a timeout of -1).

Another script connects to the default host of the web server and gets the apache status (result this time is things like the incrementing requests counter we turn into a delta using Zabbix).