Better monitoring for linux mountpoints

[svenw]

I had the problem that by only monitoring a defined set of mountpoints, repeatedly some other random ones went full without me noticing. also by monitoring a bunch of mountpoints zabbix wastes a lot of space in the database by saving redundant data (if /home is no seperate device, it just records the / data instead). so i wrote up a new check that returns ZBX_NOTSUPPORTED if a space-check is issued for a non-mountpoint directory.

here are the userchecks:

UserParameter=user.vfs.fs.size.overview,echo -e "Size\tUsed\tAvail\tUse%\tMountpoint" && df -h -P|grep ^/dev/|sed 's/^[^ ]\+ \+//g'|sed 's/\ \+/\t/g'
UserParameter=user.vfs.fs.size.fullest.info,df -Ph|sed 's/%//g'|grep ^/dev/|awk '$5 > max {max=$5;total=$2;free=$4;mount=$6} END{print max " % ("free" of "total " free on " mount ")"}'
UserParameter=user.vfs.fs.size.fullest.percent,df -P|cut -d% -f1|grep ^/dev/|awk '$5 > max {max=$5} END{print max}'

UserParameter=user.vfs.fs.inode.overview,echo -e "Size\tUsed\tAvail\tUse%\tMountpoint" && df -h -Pi|grep ^/dev/|sed 's/^[^ ]\+ \+//g'|sed 's/\ \+/\t/g'
UserParameter=user.vfs.fs.inode.fullest.info,df -Pih|sed 's/%//g'|grep ^/dev/|awk '$5 > max {max=$5;total=$2;free=$4;mount=$6} END{print max " % ("free" of "total " free on " mount ")"}'
UserParameter=user.vfs.fs.inode.fullest.percent,df -Pi|cut -d% -f1|grep ^/dev/|awk '$5 > max {max=$5} END{print max}'

UserParameter=user.vfs.fs.size[*],df -PB1 $1 |grep $1|sed 's/  */ /g'|sed 's/%//g'|cut -d" " -f$2 |egrep [1-90] || echo ZBX_NOTSUPPORTED

sadly this does not directly support same syntax as vfs.fs.size[/home,free], so i use user.vfs.fs.size[/home,{$USED}] with the following variables:

	{$TOTAL} 	 ⇒  	2
	{$USED} 	 ⇒  	3

pfree, free, pused would also be available by the col-number of df output, but this is quiet redundant

so whenever user.vfs.fs.size.fullest.percent reaches a certain level (in which i recommend to use macros aswell) i attach user.vfs.fs.size.overview to the generated mail, or at least user.vfs.fs.size.info

i'll attach a simple windows script for the fullest thing later.

there is some things way from perfect here: the selection of the "important" lines of df could maybe better be done by specifieing the fs type (-text3 -text4 -treiserfs) with df. also the echo -e is necessary for some zabbixagents while others put the -e into the output - maybe it should best be grepped away in the end, but who cares....

oh, and please mind, if one of your mountpoints reaches HIGH state trigger, and you decide to ignore it because it is ok, you wont be notified for another mountpoint, unless it exceeds the filling level of the first one.

[cdave]

What is $1 in UserParameter.. I guess i am still noob with zabbix..


UserParameter=user.vfs.fs.size[*],df -PB1 $1 |grep $1|sed 's/ */ /g'|sed 's/%//g'|cut -d" " -f$2 |egrep [1-90] || echo ZBX_NOTSUPPORTED

[marcel]

the first parameter in item[$1,$2,$3]

[mushero]

Not sure I fully understand - we do something partly similar since we have so many rapidly-changing hosts with any possible config.

First, we have a user/script that checks for the smallest space left file system and returns the space and name, like "98% /var" which zabbix parses to alert > 90% and print the name; very useful when we have no hope of getting everything checked.

But we also have a script that returns all mounted file systems, and then a huge SQL report that checks that against the file systems we are checking to report servers where we check non-mounted FS and where we don't check mounted FS.

Not great, but 100% accurate on thousands of hosts; and we keep trying to improve things.