ESXi 4.0 Hardware and Software Monitoring VMWare

[dima_dm]

I use software:
ESXi 4.0:
ESXi 4 with HP agents

https://h20392.www2.hp.com/portal/swdepot/displayProductsList.do?category=VIRTVMW

On Zabbix Server:
Software monitoring
VMware Infrastructure (VI) Perl Toolkit

http://communities.vmware.com/community/developer/forums/vsphere_sdk_perl

See vSphere SDK for Perl Installation Guide http://www.vmware.com/support/developer/viperltoolkit/

Hardware monitoring
WBEM wbemcli

http://sourceforge.net/apps/mediawiki/sblim/index.php?title=Wbemcli

Used TCP ports:
WBEM https (TCP 5989)
VMware Infrastructure (VI) Perl Toolkit https (TCP 443)

/etc/zabbix/zabbix_server.conf
…
Timeout=30
….

/etc/zabbix/zabbix_agentd.conf
Timeout=30

### VMWare WBEM Monitoring
UserParameter=WBEM[*],/etc/zabbix/zabbix_wbem_vmware.pl $1 $2 $3
UserParameter=NumericSensor[*],egrep -i "SMX_NumericSensor_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=Temperature[*],egrep -i "Temperature_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=Power_Supply[*],egrep -i "Power_Supply_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=Fan[*],egrep -i "Fan_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=Proc[*],egrep -i "Proc_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=Array[*],egrep -i "Array_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=System_Memory[*],egrep -i "System_Memory:" /tmp/vmware_$1.txt|sed 's/.*\t//'
### End VMWare WBEM Monitoring
### VMWare Perl API Monitoring
UserParameter=API[*],/etc/zabbix/zabbix_perl_api_vmware.pl $1 $2 $3
UserParameter=Volume.Capacity[*],egrep -i "Volume.Capacity_$2:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=Volume.Free[*],egrep -i "Volume.Free_$2:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=CpuTotal[*],egrep -i "CpuTotal:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=CpuUsed[*],egrep -i "CpuUsed:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=MemSize[*],egrep -i "MemSize:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=MemUsage[*],egrep -i "MemUsage:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=MaintenanceMode[*],egrep -i "MaintenanceMode:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=MEM[*],egrep -i "MEM_$2:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=Array_stat[*],egrep -i "Array_$2.$3:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=NET[*],egrep -i "vmnic$2.$3:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=Uptime[*],egrep -i "Uptime:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
### End VMWare Perl API Monitoring


For monitoring hardware I use script
/etc/zabbix/zabbix_wbem_vmware.pl IP password username

For monitoring software I use script
/etc/zabbix/zabbix_perl_api_vmware.pl IP password username

where
IP – IP address of ESXi hypervisor
password – password for ESXi hypervisor
username - username for ESXi hypervisor

The scripts writes data in files
WBEM: /tmp/vmware_IP.txt
VMware Infrastructure (VI) Perl Toolkit: /tmp/vmware_api_IP.txt
Data format:
Key: Data

IP – IP address of ESXi hypervisor

Example:
cat /tmp/vmware_172.16.38.103.txt
SMX_NumericSensor_18: 2
Temperature_5: 34
Temperature_19: 42
SMX_NumericSensor_27: 2
Power_Supply_2: 2
Temperature_6: 51
Temperature_20: 48
SMX_NumericSensor_1: 2
Temperature_18: 42
Temperature_9: 37
SMX_NumericSensor_0: 2
Proc_1: 2

Power_Supply[172.16.38.103,1] – status 1-st Power_Supply on ESXi HyperVisor 172.16.38.103
/usr/local/bin/zabbix_get -s localhost -k Power_Supply[172.16.38.103,1]
Proc[172.16.38.103,2] - status 2-nd Processor on ESXi HyperVisor 172.16.38.103
/usr/local/bin/zabbix_get -s localhost -k Proc[172.16.38.103,2]




In Zabbix template
Macros
{$HOST} - IP address of ESXi hypervisor
{$PASSWORD} – password for ESXi hypervisor
{$USERNAME} - username for ESXi hypervisor

In Zabbix HOST IP Address = 127.0.0.1 for ALL ESXi hypervisor

Testing on HP DL360G5, HP DL360G6, HP DL360G7 and ESXi 4.0, ESXi 4.1, ESXi 5.0
Use command-line utils for testing
WBEM:
/usr/bin/wbemcli ei -noverify "https://root:[email protected]/root/hpq:SMX_FAN"

VMware Infrastructure (VI) Perl Toolkit:
/usr/lib/vmware-viperl/apps/performance/viperformance.pl --host vmware2 --server 172.16.38.103 --countertype net --samples 1 --instance "*" --username root --password XXXXXX

Alternative method of access to WBEM Class via VMware Infrastructure (VI) Perl Toolkit

http://communities.vmware.com/docs/DOC-10665

1.10.2010 New Version ESXi4_VMWare.2010.10.01.zip
Changes:
1) Error handler for wbemcli is improved
Administration ->General ->VALUE MAPPING for Items "Perl API client status" and "WBEM client status"
VMWare client status

0 OK
1 VMWare client error
2 Status file write error
3 bad username or password
4 Can't connect to ESXi
5 Can't get data from ESXi

2)Template for ESXi4.1 is added
3) add -i option for egrep in /etc/zabbix/zabbix_agentd.conf

16.11.2010 new XML Template
P.S. On other versions ESXi the name of variables can be others. Fix names in Template Template_VMWare_HP_DL360_G6.xml

[alubel]

I have a similar need: To monitor hardware health of ESXi box.

my only difference is im running sun 4170's.

The approach I took was to use IPMI directly to the Service processor. good thing about zabbix is that to do both host monitoring and IPMI monitoring, you dont have to create 2 hosts

the advantage of doing it out of band is that if a server goes down, we have logs and graphs of the events that led up to it.

What would be more interesting is hacking up the good old nagios check_esx scripts that use perl sdk and report things like disk I/O latency etc to zabbix

[dima_dm]

Disk I/O you can see in the script zabbix_perl_api_vmware.pl (Array_stat)

[Murilex]

The idea of setting the IPs of my esxi boxes to 127.0.0.1 didn't sound very good for me. How about use your scripts as external check items? This way, we didn't need to use macros for real IPs of esxi boxes and these IPs would be passed automatically by zabbix server as parameters to external scripts.

[dima_dm]

Monitoring scripts it is polling a lot of variables, and use directly External Check will create very much unwanted loading of Zabbix and WMWare server.
You can use method:
To start polling of variables from cron (or External Check) and to write data to temp files.
External Check script will be get data from temp files

I register ESXi Hupervisor and ESXi guest system as separate monitoring objects and 127.0.0.1 in Zabbix does not create for me any problems.

[Murilex]

Hello dima_dm! I haven't understood very well what you said . My plans are use your main scripts (zabbix_perl_api_vmware.pl and zabbix_wbem_vmware.pl) as external monitoring items. This way I could use real IPs of my ESXi hypervisors (zabbix hosts) and not fake IPs (127.0.0.1). I wouldn't change the logic of your scripts, therefore, they would continue to write info in temp files. I intend to write some scritps in order to read the temp files. These scripts would also be run by zabbix server as external monitoring scripts, again passing the real IP of ESXi hypervisors as parameters.

I wonder, considering that zabbix server and zabbix agents processes are running on the same machine, are there any benefits (for the server load) of using your scripts as user parameters (in zabbix agents) instead of as external monitoring items?

PS: I said nothing about monitoring of VMware guests.

[dima_dm]

You have understood correctly.
Productivity will be identical with External check (get data via temp files) and Zabbix Agent.

[Tristan]

why don't you just send the output from the perl scripts with the zabbixsender perl version to zabbix?

[dima_dm]

Because to debug and solve problems is much easier. At once it is clear where a problem in Zabbix or script.

[dima_dm]

1.10.2010 New Version ESXi4_VMWare.2010.10.01.zip

http://www.zabbix.com/forum/showpost.php?p=58726&postcount=1

Changes:
1) Error handler for wbemcli is improved
2) Template for ESXi4.1 HP DL360G6 is added
3) add -i option for egrep in /etc/zabbix/zabbix_agentd.conf

[saintxseiya]

Is there any way to monitor non HP or at least HPs that do not have agents on their system ?

I am not able to do that with your scripts.

Anyone could help?

[dima_dm]

Is there any way to monitor non HP or at least HPs that do not have agents on their system ?

I am not able to do that with your scripts.

Anyone could help?

HP agents is used only for Hardware monitoring.
You can use IPMI via HP iLO.
You can install HP agents
ftp://ftp.hp.com/pub/softlib2/softwa...bundle-1.1.zip
or
ftp://ftp.hp.com/pub/softlib2/softwa...bundle-1.4.zip
Instruction in Attach.
For ESXi4.1

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?swItem=MTX-b98795300e7947d88f19ab56d6&lang=en&cc=us&idx=1&mode=4&

[saintxseiya]

Hey thanks dima,

sorry i was on holidays for one week, so with the package, i will be able to monitor the ESXi also if it are no HP machines or not supported ones, is that correct?

Many Thanks in advance!

[dima_dm]

Hey thanks dima,

sorry i was on holidays for one week, so with the package, i will be able to monitor the ESXi also if it are no HP machines or not supported ones, is that correct?

Many Thanks in advance!

No. This agents is used only for HP. Agents are rigidly linked to Hardware. You should receive agents from the your Hardware vendor.
Or you can use alternative methods of Hardware monitoring, which your Hardware vendor is supported (IPMI etc)

[Toepfer]

Hi, I am new to Zabbix and have tried to implement the ESXi monitoring as follows:

1) Zabbix Server 1.8.3 installed on CentOS 5.5

2) ESXi Server 4.1 with HP Management Agents incl. WBEM (I need WBEM for other applications)

3) centOS (Zabbix Server) I have installed wbemcli

4) On CentOS I have entered the follwowing command using the right IP and user data of the ESXi Server:
/usr/bin/wbemcli ei -noverify "https://root:[email protected]/root/hpq:SMX_FAN"

The result was OK, I have retrieved Data from ESXi.

5) I have added the following code to the end of my CentOS Zabbix Server zabbix_agent.conf

Timeout=30

### VMWare WBEM Monitoring
UserParameter=WBEM[*],/etc/zabbix/zabbix_wbem_vmware.pl $1 $2 $3
UserParameter=NumericSensor[*],egrep -i "SMX_NumericSensor_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=Temperature[*],egrep -i "Temperature_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=Power_Supply[*],egrep -i "Power_Supply_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=Fan[*],egrep -i "Fan_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=Proc[*],egrep -i "Proc_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=Array[*],egrep -i "Array_$2:" /tmp/vmware_$1.txt|sed 's/.*\t//'
UserParameter=System_Memory[*],egrep -i "System_Memory:" /tmp/vmware_$1.txt|sed 's/.*\t//'
### End VMWare WBEM Monitoring
### VMWare Perl API Monitoring
UserParameter=API[*],/etc/zabbix/zabbix_perl_api_vmware.pl $1 $2 $3
UserParameter=Volume.Capacity[*],egrep -i "Volume.Capacity_$2:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=Volume.Free[*],egrep -i "Volume.Free_$2:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=CpuTotal[*],egrep -i "CpuTotal:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=CpuUsed[*],egrep -i "CpuUsed:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=MemSize[*],egrep -i "MemSize:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=MemUsage[*],egrep -i "MemUsage:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=MaintenanceMode[*],egrep -i "MaintenanceMode:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=MEM[*],egrep -i "MEM_$2:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=Array_stat[*],egrep -i "Array_$2.$3:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=NET[*],egrep -i "vmnic$2.$3:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
UserParameter=Uptime[*],egrep -i "Uptime:" /tmp/vmware_api_$1.txt|sed 's/.*\t//'
### End VMWare Perl API Monitoring

6) I have copied the files to the follwoing location:
/etc/zabbix/zabbix_wbem_vmware.pl
/etc/zabbix/zabbix_perl_api_vmware.pl

7) I have edited both .pl files, changes: IP, username, passord of ESXi Server.

8) Restart of zabbix server and agent on CentOS

9) Import of Template to Zabbix: Template_VMWare_HP_DL360_G6_ESXi4.1.xml
Error:
zabbix Not allowed symbols or sequence of symbols in expression element detected
and a lot of code in red



What did I wrong?
Could someone provide an explanation including all steps for ESXi WBEM montitoring.


Thx