VmBix 1.2.1 released

[tatapoum]

Hi,

I've released a new version of VmBix (https://github.com/dav3860/vmbix/). It includes some nice new features compared to 1.1.x :
- VM, ESX hosts and datastore objects are cached for better performance.
- it's possible to query the objects using their UUID.
- I've added a Zabbix loadable module based on zabbix_get to query VmBix with great performance (tested on 2.2 and 2.4), while still allowing the virtual machines to use an agent.
- etc.

You can find the full CHANGELOG here : https://github.com/dav3860/vmbix/blo...r/CHANGELOG.md.

And you can download the latest release here : https://github.com/dav3860/vmbix/releases.

[whosgonna]

Unit of Measurement

It looks like all values returned are integers, when perhaps some should be float values. I don't know if this is a characteristic of VmBix, or the VMware API. For example, the following command:

zabbix_get -s 127.0.0.1 -p 12050 -k esx.counter.list[host01.domain.local]

has it's first value returned as

2 : cpu.usage : Usage in Percent (rate,average)

Based on this, I run the following command expecting a percentage returned.

zabbix_get -s 127.0.0.1 -p 12050 -k esx.counter[host01.domain.local,cpu.usage]

and end up with a returned value of 2191 as I write this. Comparing this to the usage reported by the host in the vSphere Client's performance view, I would guess that this would be 21.91 (%), but I'd prefer to understand what I'm looking at, rather than relying on my guessing.

Finally, is there an explanation of how to use the interval value in these queries? I expect this would rely more on the VMware documentation, but it would be nice to know specifically how this impacts the returned values.

[tatapoum]

The integer/float thing comes from the VMWare API. The returned values are Long types, so it's needed to process them afterwards (/100 in this case). I'm going to add this to VmBix when I have time. It makes sense.

The counter method parameters are :

vm.counter[(uuid|name),counter,[instance,interval]]

The instance is related to the optional instance name of the metric. Typically if you have a few vdisks, you can query the overall vdisk write usage using :

TEST ~ # zabbix_get -s 127.0.0.1 -p 12050 -k vm.counter[VM01,virtualDisk.write]
927

Or get the vdisk list with:

TEST ~ # zabbix_get -s 127.0.0.1 -p 12050 -k vm.counter.discovery[VM01,virtualDisk.write]
{
    "data": [
        {
            "{#METRICINSTANCE}": "scsi0:2"
        },
        {
            "{#METRICINSTANCE}": "scsi0:0"
        },
        {
            "{#METRICINSTANCE}": "scsi0:1"
        },
        {
            "{#METRICINSTANCE}": ""
        }
    ]
}

Finally, you can query each vdisk independently :

TEST ~ # zabbix_get -s 127.0.0.1 -p 12050 -k vm.counter[VM01,virtualDisk.write,scsi0:1]
822

Regarding the interval, VmBix default value is 300s. The behavior of the VMWare API is that you have two kinds of intervals ;
- historical intervals (average/sum over 5mn, 30mn, etc) -> these intervals are predefined.
- real-time

With Zabbix, we need to query the performance values every X seconds. The historical intervals would be great for this but there is a drawback. If you connect to the vCenter instead of a ESX, the values are aggregated over 30mn at a minimum, which isn't very precise. We cannot directly use the real-time intervals every 300s for example, as it's just a real-time value.
So what I've done in VmBix is that it collects the real-time values over the defined interval period, and aggregate them afterwards.

TEST ~ # zabbix_get -s 127.0.0.1 -p 12050 -k vm.counter.list[VM01] | grep virtualDisk.write
327 : virtualDisk.write : Write rate in Ko/s (rate,average)
TEST ~ # zabbix_get -s 127.0.0.1 -p 12050 -k vm.counter[VM01,virtualDisk.write,scsi0:1,60]
728

As it's a rate, VmBix returns the average rate over 60s.

TEST ~ # zabbix_get -s 127.0.0.1 -p 12050 -k vm.counter.list[VM01] | grep net.packetsRx
113 : net.packetsRx : Received packets in Number (delta,summation)
TEST ~ # zabbix_get -s 127.0.0.1 -p 12050 -k vm.counter[VM01,net.packetsRx,,120]
16865

As it's a delta, VmBix returns the sum over 120s.


So you need to set the interval to the Zabbix polling interval to get consistent values.

[tatapoum]

I've released a version 1.2.2, which fixes the "percent" issue.

[bobmarkinson]

Excellent! It's working perfectly in my Zabbix 2.4 environment. Updated to 1.2.2 without issues. I really like the flexibility that VmBix offers when it comes to being able to query any performance counter which the built in VMware can't do.

[whosgonna]

Thank you again

Thank you very much for the feedback, as well as the wonderful method of monitoring VMware.

With 1.1.2, you say you fixed the percentages. I'm guessing that this means I'll have to change my data types from numeric to float if I update. Can you confirm?

[tatapoum]

No. I decided to convert the value to an integer (32,45% -> 32%). Don't forget to remove the multiplier.

[whosgonna]

Do you think there's a need to worry about loss of granularity? I think that most of the items that i'm looking for tenth and hundredth of a percent on are calculated items (ready%, costop%, etc), so those should retain the same level of precision. Am I over-thinking this?

[barbaros]

Guys, I need some help with compiling VmBix. On CentOS 6.5 I get:

compile:
[echo] Compiling...
[javac] Compiling 1 source file to /root/vmbix/build/classes
[javac] error: error reading /root/vmbix/lib/guava.jar; error in opening zip file
[javac] 1 error

BUILD FAILED
/root/vmbix/build.xml:12: Compile failed; see the compiler error output for details.

Total time: 5 seconds

The file mentioned is a little bit strange .

([email protected] lib)# cat guava.jar
<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>nginx/0.8.55</center>
</body>
</html>

I am installing VmBix through proxy:

git config --global http.proxy http://proxy.local:3128
git config --global https.proxy http://proxy.local:3128
export ANT_OPTS="-Dhttp.proxyHost=proxy.local -Dhttp.proxyPort=3128 -Dhttps.proxyHost=proxy.local -Dhttps.proxyPort=3128"
(...)

EDIT:
Strange, manually I can download the problematic file:

http://search.maven.org/remotecontent?filepath=com/google/guava/guava/18.0/guava-18.0.jar

EDIT 2:
Without proxy (with direct traffic to the Internet) the same error.

EDIT 3:
OK, I've successfully used the precompiled sources. BTW, rmp-based should be rpm-based, I think .

[barbaros]

tatapoum, is there possibility to redesign templates in a way, that datastores will appear under the specific ESX host, not under vCenter Server? Similar to the native vSphere monitoring from Zabbix.

BTW, great work, thanks!

[karltlg]

Discovered this module today, very good.

But I am having problems with compiling the "simple check" module. I downloaded the sources and did everything according to manual, but getting error while starting compiling

gcc -shared -o vmbix.so vmbix-2.4.c -I../../../include -fPIC
In file included from ../../../include/common.h:23:0,
                 from vmbix-2.4.c:23:
../../../include/sysinc.h:23:20: fatal error: config.h: No such file or directory
 #include "config.h"
                    ^
compilation terminated.
make: *** [vmbix-2.4] Error 1

When I navigate do the include directory, I cannot find config.h file there either.

Running Ubuntu 14.04

[danrog]

I've released a version 1.2.2, which fixes the "percent" issue.

I've taken all of my custom additions and added them to your version 1.2.2. How should I get them over to you?

Some of the things I added:
1. If you are querying an ESX host datastore, I return the datastore name and the instance so you can actually add the 'display' name to your items (sort of a hack but there isn't an easy way to get the display name). I should note, it only works for VMFS based stores (SAN or iSCSI based, something else has to be done for NFS/NAS based stores)

2. Added alert name mappings to the alerts themselves. Example:
red: Host connection and power state
yellow: Host connection state
red: vSphere Distributed Switch VLAN trunked status
red: vSphere Distributed Switch MTU supported status

3. The getLatestEvents function retrieves events over the last five minutes and includes username etc...

4. Retrieve hardware based issues: Disk, Memory, CPU

5. Retrieve inventory based info: Serial number, Vendor, Model, OS, OS Short

6. Fixed small bug in the datastore uncommitted return value

[waclaw]

Hi,

I also added some features, and created github pull request to tatapoum (dav3860)
You can check my changes on github: https://github.com/gites/vmbix


Added features:
* cluster checks
* some more info in datastores discovery ({#UUID}, {#CLUSTER}, {#LOCAL}, {#NAA}
* some options to configure cache TTL and size

[tatapoum]

danrog, could you rebase your changes with the last version and create a pull request on https://github.com/dav3860/vmbix ?

I've taken all of my custom additions and added them to your version 1.2.2. How should I get them over to you?

Some of the things I added:
1. If you are querying an ESX host datastore, I return the datastore name and the instance so you can actually add the 'display' name to your items (sort of a hack but there isn't an easy way to get the display name). I should note, it only works for VMFS based stores (SAN or iSCSI based, something else has to be done for NFS/NAS based stores)

2. Added alert name mappings to the alerts themselves. Example:
red: Host connection and power state
yellow: Host connection state
red: vSphere Distributed Switch VLAN trunked status
red: vSphere Distributed Switch MTU supported status

3. The getLatestEvents function retrieves events over the last five minutes and includes username etc...

4. Retrieve hardware based issues: Disk, Memory, CPU

5. Retrieve inventory based info: Serial number, Vendor, Model, OS, OS Short

6. Fixed small bug in the datastore uncommitted return value

[Zabbix_User_1138]

Problem With ESXi Host Metrics

When looking at the cpu.ready metric for an ESXi host, the values collected in Zabbix don't match up the the values reported in vCenter.
The hourly host server average in the real-time vCenter view (20 sec interval) is reported as roughly 27ms, but the same average in Zabbix (60 sec interval) is at 54ms, or 2x up the vCenter value.
I would expect it to be 3x up the vCenter value.

The CPU Ready item key is vmbix[esx.counter,{HOST.HOST},cpu.ready,,60]
The item update interval is set to 60 sec and the VmBix config interval is also set to 60 sec. Not sure if the VmBix config interval is needed to be at 60 sec to produce proper values, but I figured it didn't hurt to change it from the default 300 sec.

Other metrics seems to be off too, so far I've found these to be different in Zabbix when compared to vCenter:
vmbix[esx.vms.memory,{HOST.HOST},active]
vmbix[esx.vms.memory,{HOST.HOST},consumed]
vmbix[esx.vms.memory,{HOST.HOST},private]
vmbix[esx.vms.memory,{HOST.HOST},shared]
vmbix[esx.vms.memory,{HOST.HOST},overheadConsumed]

But its not all the metrics that are off, vmbix[esx.memory,{HOST.HOST},used] and vmbix[esx.cpu.load,{HOST.HOST},used] are spot on, to name a few.

I'm running Zabbix 2.4.3 and VmBix 1.2.2 on CentOS 6.5