zabbix 2.0 and vfs.dev.read (write)

[ghillan]

Hello,
im having a problem triing to get I/O from the FS mounted on linux servers. I just added another item prototype to the existing low level discovery in the template "Template OS linux" but the items become usupported.

Tryng to get manually the request from the command line i get:

/opt/zabbix/bin/zabbix_get -s SERVERNAME -k vfs.dev.read[/tmp]
ZBX_NOTSUPPORTED

but the other "vfs" items works correctly. For example:
/opt/zabbix/bin/zabbix_get -s SERVERNAME -k vfs.fs.size[/tmp]
46139891712

Odd enough if i use the real device name ( not the moutpoint) the result change:

/opt/zabbix/bin/zabbix_get -s SERVERNAME -k vfs.dev.read[/dev/mapper/rootvg-root]
0.000000 ( the result its alwasys 0.000 on any mount point, so probably it does'nt work either, butr at least im not getting the "ZBX_NOTSUPPORTED")

So for some reasons seems that those 2 specific items have problems if you pass {#FSNAME} value from the low level discovery.

I did something wrong or i really discovered a bug?

Any advice even for a workaound would be really apprecyated. I need to monitor disk IO for my san-attached LUN's. If there is any other way i would use it too.

tnx in advance.

[ahowell]

You can't use mount points as a parameter for vfs.dev.*
Try setting the type specifically.
E.g.

vfs.dev.write[/dev/mapper/rootvg-root,operations]

Or

vfs.dev.write[/dev/mapper/rootvg-root,sectors]

[ghillan]

You can't use mount points as a parameter for vfs.dev.*
Try setting the type specifically.
E.g.

vfs.dev.write[/dev/mapper/rootvg-root,operations]

Or

vfs.dev.write[/dev/mapper/rootvg-root,sectors]

this way seems working, but i cannot manually add all items for each hosts!

I wanted to add another prototype in the low level discovery, so the key should be something like:

vfs.dev.read[{#FSNAME}, sectors]

But the discovery traslate {#FSNAME} as "/", "/tmp" and so on. Other item prototype works perfectly. For example:

vfs.dev.read[{#FSNAME}, sectors]

There are other macros that return the devices instead the mount points?


I did something similar with snmp discovery where you have "SNMPVALUE" and "SNMPINDEX", but for FS i see only FSNAME.

[ahowell]

this way seems working, but i cannot manually add all items for each hosts!

I wanted to add another prototype in the low level discovery, so the key should be something like:

vfs.dev.read[{#FSNAME}, sectors]

But the discovery traslate {#FSNAME} as "/", "/tmp" and so on. Other item prototype works perfectly. For example:

vfs.dev.read[{#FSNAME}, sectors]

There are other macros that return the devices instead the mount points?


I did something similar with snmp discovery where you have "SNMPVALUE" and "SNMPINDEX", but for FS i see only FSNAME.

You need to write a custom LLD script to return the device names instead.

I've attached one that we use (it needs to be made suid root) which discovers LVM devices.

Add a UserParameter for this script to your agents, then you can create a new LLD which uses this. We use vfs.dev.discovery as the name.

[ghillan]

Tnx for the help.

I was hoping to make them work without external scripts, but at this point i changed my plans and made something even more focused on my needs.

I have to monitor also SAN LUNS used as raw devices by several oracleasm machines. So i made a script that parses all " dm-* " lines in /proc/discstats and returned the results to zabbix in JSON format.

I made a template with prototypes and it works like a charm.

[bonobo_slr]

Hi - I am trying to do the same thing to auto-detect the devices for vfs.dev discovery. I was just wondering what it is I must do with the script?

I am not sure where I must put it and how I can use it.

Thanks.

[ghillan]

suppose you want to monitor a server called "myserver". You have to install a zabbix agent there. so you have a "zabbix_agentd.conf" to edit.

You have to add a line in the conf file: for example:
UserParameter=device_mapper[*],/usr/local/zabbix/bin/device_mapper.pl $1 $2



where "device_mapper.it" is the script i have done and accept 2 parameters: the "dm-* device and the info adked ( queue, wtrites, reads, and so on )":



my script:

################################################
#!/usr/bin/perl

$first = 1;

$dm_device = $ARGV[0];
$dm_info = $ARGV[1];

if (!$dm_device) {
$exec = "cat /proc/diskstats | grep \"dm-\"";
} else {
$exec = "cat /proc/diskstats | grep \"$dm_device\""
}

if (!$dm_info) {
print "{\n";
print "\t\"data\":[\n\n";

for (`$exec` ) {
( $major_number,
$minor_number,
$dm_name,
$reads_completed,
$reads_merged,
$reads_sectors,
$reads_time,
$writes_completed,
$writes_merged,
$writes_sectors,
$writes_time,
$io_queue,
$io_time,
$weighted_io_time
) = m/ +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+)/;

print "\t\n" if not $first;
$first = 0;

print "\t\t{";
print " \"{#DMNAME}\":\"$dm_name\"";
print "\t},";
}
print "\n\t]\n";
print "}\n";
} else {

for (`$exec` ) {
( $major_number,
$minor_number,
$dm_name,
$reads_completed,
$reads_merged,
$reads_sectors,
$reads_time,
$writes_completed,
$writes_merged,
$writes_sectors,
$writes_time,
$io_queue,
$io_time,
$weighted_io_time
) = m/ +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+)/;

}
if ($dm_info eq "MAJOR") {print $major_number;}
elsif ($dm_info eq "MINOR") {print $minor_number;}
elsif ($dm_info eq "DMNAME") {print $dm_name;}
elsif ($dm_info eq "READS_COMPLETED") {print $reads_completed;}
elsif ($dm_info eq "READS_MERGED") {print $reads_merged;}
elsif ($dm_info eq "READS_SECTORS") {print $reads_sectors;}
elsif ($dm_info eq "READS_TIME") {print $reads_time;}
elsif ($dm_info eq "WRITES_COMPLETED") {print $writes_completed;}
elsif ($dm_info eq "WRITES_MERGED") {print $writes_merged;}
elsif ($dm_info eq "WRITES_SECTORS") {print $writes_sectors;}
elsif ($dm_info eq "WRITES_TIME") {print $writes_time;}
elsif ($dm_info eq "IO_QUEUE") {print $io_queue;}
elsif ($dm_info eq "IO_TIME") {print $io_time;}
elsif ($dm_info eq "WIO_TIME") {print $weighted_io_time;}
else {print "ERROR: Chose one of the following info:\n MAJOR MINOR DMNAME READS_COMPLETED READS_MERGED READS_SECTORS READS_TIME WRITES_COMPLETED WRITES_MERGED WRITES_SECTORS WRITES_TIME IO_QUEUE IO_TIME WIO_TIME\n";}
}

################################################

if executed without parameters it return the list of dm's:

{
"data":[

{ "{#DMNAME}":"dm-0" },
{ "{#DMNAME}":"dm-1" },
{ "{#DMNAME}":"dm-2" },
{ "{#DMNAME}":"dm-3" },
{ "{#DMNAME}":"dm-4" },
{ "{#DMNAME}":"dm-5" },
{ "{#DMNAME}":"dm-6" },
{ "{#DMNAME}":"dm-7" },
{ "{#DMNAME}":"dm-8" },
{ "{#DMNAME}":"dm-9" },
{ "{#DMNAME}":"dm-10" },
{ "{#DMNAME}":"dm-11" },
{ "{#DMNAME}":"dm-12" },
{ "{#DMNAME}":"dm-13" },
{ "{#DMNAME}":"dm-14" },
{ "{#DMNAME}":"dm-15" },
{ "{#DMNAME}":"dm-16" },
{ "{#DMNAME}":"dm-17" },
{ "{#DMNAME}":"dm-18" },
]
}

i use it to make the discovery. then i created an item for any item, and the same script will be used with 2 parameter ( dmname and the info you want for that item).
last thing to do its cheate the item:

I created a template ad made a discovery item for each paramteter:
example:
IO queue on {#DMNAME} device_mapper[{#DMNAME},IO_QUEUE]

exactly the same for the other infos:

MAJOR
MINOR
DMNAME
READS_COMPLETED
READS_MERGED
READS_SECTORS
READS_TIME
WRITES_COMPLETED
WRITES_MERGED
WRITES_SECTORS
WRITES_TIME
IO_QUEUE
IO_TIME WIO_TIME

basically all the info included in the diskstats file. You might want all or part of them, but the script can get any.


Hope it helped. it took my a while to understand how to do it, but the manual its quite helpfull.

[neilb]

I created a template ad made a discovery item for each paramteter:
example:
IO queue on {#DMNAME} device_mapper[{#DMNAME},IO_QUEUE]

exactly the same for the other infos:

MAJOR
MINOR
DMNAME
READS_COMPLETED
READS_MERGED
READS_SECTORS
READS_TIME
WRITES_COMPLETED
WRITES_MERGED
WRITES_SECTORS
WRITES_TIME
IO_QUEUE
IO_TIME WIO_TIME

basically all the info included in the diskstats file. You might want all or part of them, but the script can get any.


Hope it helped. it took my a while to understand how to do it, but the manual its quite helpfull.

hi ghillan

any chance of grabbing a copy of your template please?

cheers
Neil

[ghillan]

I cannot connect to that server anymore, so i cannot get that template i've done.

Sorry

[neilb]

ok thanks anyway!

Neil.

[pietro54]

neilb, can I please you for a little modification, I need to monitor disk like sda, sdb etc. in older OS. I believe that is really little modification of your script, maybe it wont be a problem for you?

[ghillan]

neilb, can I please you for a little modification, I need to monitor disk like sda, sdb etc. in older OS. I believe that is really little modification of your script, maybe it wont be a problem for you?

I made that script for my use, but posted here to avoid other people to think from scratch how to do the same/similar thing. If you want to modify/develop/upgrade to make it more suitable for your usage, you are free to do so.

[pietro54]

Hello,
Me and me koleg, we made some change in this script.
There is new "if"
so if script will find some lvm called dm- then he will generate statistic for them, but if he wont he will be searching for standard partitions like "sd"

Now script looks like:

################################################
#!/usr/bin/perl

$first = 1;

$dm_device = $ARGV[0];
$dm_info = $ARGV[1];

$exec = "cat /proc/diskstats | grep \"dm-\"";
$c = "$exec | wc -l";

if (`$c` < 1) {
$exec = "cat /proc/diskstats | grep \"sd\"";
}


if (!$dm_info) {
print "{\n";
print "\t\"data\":[\n\n";

for (`$exec` ) {
( $major_number,
$minor_number,
$dm_name,
$reads_completed,
$reads_merged,
$reads_sectors,
$reads_time,
$writes_completed,
$writes_merged,
$writes_sectors,
$writes_time,
$io_queue,
$io_time,
$weighted_io_time
) = m/ +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+)/;

print "\t\n" if not $first;
$first = 0;

print "\t\t{";
print " \"{#DMNAME}\":\"$dm_name\"";
print "\t},";
}
print "\n\t]\n";
print "}\n";
} else {

for (`$exec` ) {
( $major_number,
$minor_number,
$dm_name,
$reads_completed,
$reads_merged,
$reads_sectors,
$reads_time,
$writes_completed,
$writes_merged,
$writes_sectors,
$writes_time,
$io_queue,
$io_time,
$weighted_io_time
) = m/ +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+) +(\S+)/;

}
if ($dm_info eq "MAJOR") {print $major_number;}
elsif ($dm_info eq "MINOR") {print $minor_number;}
elsif ($dm_info eq "DMNAME") {print $dm_name;}
elsif ($dm_info eq "READS_COMPLETED") {print $reads_completed;}
elsif ($dm_info eq "READS_MERGED") {print $reads_merged;}
elsif ($dm_info eq "READS_SECTORS") {print $reads_sectors;}
elsif ($dm_info eq "READS_TIME") {print $reads_time;}
elsif ($dm_info eq "WRITES_COMPLETED") {print $writes_completed;}
elsif ($dm_info eq "WRITES_MERGED") {print $writes_merged;}
elsif ($dm_info eq "WRITES_SECTORS") {print $writes_sectors;}
elsif ($dm_info eq "WRITES_TIME") {print $writes_time;}
elsif ($dm_info eq "IO_QUEUE") {print $io_queue;}
elsif ($dm_info eq "IO_TIME") {print $io_time;}
elsif ($dm_info eq "WIO_TIME") {print $weighted_io_time;}
else {print "ERROR: Chose one of the following info:\n MAJOR MINOR DMNAME READS_COMPLETED READS_MERGED READS_SECTORS READS_TIME WRITES_COMPLETED WRITES_MERGED WRITES_SECTORS WRITES_TIME IO_QUEUE
IO_TIME WIO_TIME\n";}
}

################################################

Hope it will be helpful.

[ZabbixFun]

Personally, I use Userparameters to monitor disk IO.
E.g.:

UserParameter=custom.vfs.dev.read.ops[*],cat /proc/diskstats | grep $1 | head -1 | awk '{print $$4}'
.....

And I have created items with keys like:

custom.vfs.dev.read.ops
custom.vfs.dev.read.ms
.....

No scripts used... just Userparameters...
Find attached the template I use and a file with the Userparameters.
The template includes 16 Items (8 for sda and 8 for sdb) and 4 graphs.

Please tell me if you find it useful.

Regards,