First of all sorry for my English. 
In my daily work, I need to monitor not only the physical machines by using standard zabbix interfaces but to catch up snmp trap-s from special vendor specific programs running on these machines. By reviewing several options for trap receiving on the wiki and the forums, I realized that the existing solutions do not satisfy my needs. Therefore, I decided to write a handler script for the snmp traps, taking as a basis for one of those that had been posted.
What my solution can:
• search the zabbix database, in order to find the hostname by IP address or the correct IP from the hostname
• exclusion file, containing lines with regular expressions, which are applied to the combined string of the received trap and when coincidence - the trap is not transferred to zabbix. This is used to remove "unnecessary" traps such as Heartbeat, etc.
• aliases file, containing values of type IP - hostname. Used in cases where the traps are coming from different IP in case of multiple NICs, or where the traps come from multiple physical devices, but "logically" it would be better to keep them in one place.
• and, most interesting part, a request is made to the zabbix database to find the right item for which the trap came to (the script finds the trap key oid which is passed by value SNMPv2-MIB:: snmpTrapOID.0). Moreover, search is done by the key, and you can create a zabbix item for the different traps, and the key may be "common" for a several traps. Example: if there is zabbix host TrapTest, and for this host the item is created (such as Zabbix Trapper) with the value key = .1.3.6.1, and the second item with the value key = .1.3.6.1.4.1, when you get the trap which is decoded to 1.3.6.1.4.1.5.6 it diverted to the second item, and when the trap is decoded to .1.3.6.1.4 it is being diverted to the first item.
Disadvantages of this script:
• for each received trap a 2 DB queryis performed, which, in the presence of a large number of hosts and a lot of traps can increase the load to the database. (Although the correct use of exclude file helps to decrease load)
• for each received trap a separate process handler is started, which may also affect the performance of the machine (but that is a common problem not only for my solution, but to the implementation of the trap handling from the net-snmp, and to solve this issue, developers of zabbix must insert into their product native support for traps instead of net-snmp snmptrapd)
How to configure:
Let me show instructions on how to configure and make all this to work on the example of CentOS 5.4 and versions of zabbix 1.8 and net-snmp-5.3.2.2-7.el5
Install the perl script in any directory (I use /usr/local/sbin/), extracted from the archive.
To start you need to have a following perl modules installed: DBI and DBD::Mysql.
For my system they are available in RPM packages perl-DBI-1.52-2.el5 iperl-DBD-MySQL-3.0007-2.el5
Edit the perl script to suit your needs:
• For beginning, I recommend to enable logging
• Everything will be logged in the following file
• Address of the zabbix server and port
• Name of the default host to which traps will be sent if zabbix will not find it from a DB. It must be the same as you define it in zabbix GUI
• If 1 then do not try to find a host from the DB and send everything to the default host (do not know why you may use it )
• Name of the default Item to which traps are send to. It must be the same as you define it in zabbix GUI
• Path to zabbix_sender
• zabbix database, IP, username and password
• The path and file name of the exclude file in which are listed regular expressions (multiple lines) that applied to the received trap, and in case of coincidence, this trap is not sent to zabbix. It is used for "screening" not necessary traps.
• The name of the file containing the lines of two fields separated by tabs such as IP<TAB>hostname. It is used if we have traps coming from a different addresses than we have a host created on zabbix (in the case of multiple network adapters)
• By default snmptrapd sends traps in the form of SNMP-COMMUNITY-MIB:: snmpTrapAddress.0. This option allows you to cut "unnecessary" parts of a trap text from the left part of "::" to reduce the overall size of the data
The machine where all traps are sent to must be correctly set up for the reception of the traps. To do this there must be running snmptrapd from the net-snmp package. At least configuration file for snmptrapd should contain:
Where in the line traphandle you should specify the full path of the perl script trap handler.
And of course the script must have execute permissions set.
Next, we need the file describing the device SNMP, the MIB file. Which contains the actual description of how and what will be accepted by SNMP. The fact that the device sends all the information (oid) in the numeric form .1.3.6.1, etc., which is not readable and understandable by human. And the MIB file contains a description of meaning of this oid.
For standard devices the MIB files can be found on the Internet, and for your specific ones – contact the equipment supplier or look on the Internet (a very nice resource - http://www.oidview.com/mibs/detail.html).
By default (at least in my case) MIB files in the system are stored in /usr/share/snmp/mibs
Copy your file to this directory.
Edit the file /etc/snmp/snmp.conf and add there a line
To find out MIB_MODULE_NAME, open the MIB file and find a line like NET-SNMP-MIB DEFINITIONS:: = BEGIN
What is located to left side before DEFINITIONS, this is the right MIB module name.
(I'm doing like this - and it is working for me. Certainly you can do another way. See the man for snmp.conf)
Restart snmptrapd service:
Now the net-snmp is ready to receive traps from your equipment.
Then adjust zabbix.
Create a host with the name as specified in our script (zabbix_snmptraphandler.pl) $WILDCARD_HOST. It is used when the trap is coming from the device which is not yet added to the zabbix. Add to that host item with type Zabbix trapper and the type of received information “Text”, with Key as specified in our script variable $ZABBIX_ITEM. This is the default name, which will received or traps if you have not set up the personal trap items for that host.
Now create hosts for your equipment, and add to them an item with type zabbix trapper and the type of received information “Text”, with Key as specified in our script variable $ZABBIX_ITEM. If we know beforehand the list of those traps that can come from some specific equipment, than create personal item with type zabbix trapper but in the Key we must specify numeric representation of a trap. To obtain such representation, run the command with a trap that we expect. For example:
In response, we obtain a line like .1.3.6.1.4.1.232.0.6048
This is a numeric representation of trap OID.
Now, if, in the created Item’s key we define .1.3.6.1.4.1.232.0.6048, then all the traps of this type/OID will be sent to this item. If we specify in the Key .1.3.6.1.4.1.232.0 then all traps that are under this OID (a list can be found by clicking snmptranslate-Tp .1.3.6.1.4.1.232.0) will be sent to this Item . Thus, we can create a "common" Items for each type of received traps.
To make our new item working properly with the triggers we have to do following. For one previously defined item we are configuring same number of the triggers as the real SNMP traps can be send for (we have to look into the MIB file). For each trigger we put expression like:
I the first str() function we should specify the unique text that is received from the particular trap to set alarm, and in the second str() function we should specify the unique text with is present in the trap text to clear already received trap correspondingly. In this case triger will work as it should, generating event proper both for Alarm and Clear, and we do not need to use nodata() anymore
For creating a lot of triggers I am using some simple scripts generating the xml files from MIBs for importing them into zabbix. But, as MIB files are not standard for trap descriptions, I can not share these scripts.
In my daily work, I need to monitor not only the physical machines by using standard zabbix interfaces but to catch up snmp trap-s from special vendor specific programs running on these machines. By reviewing several options for trap receiving on the wiki and the forums, I realized that the existing solutions do not satisfy my needs. Therefore, I decided to write a handler script for the snmp traps, taking as a basis for one of those that had been posted.
What my solution can:
• search the zabbix database, in order to find the hostname by IP address or the correct IP from the hostname
• exclusion file, containing lines with regular expressions, which are applied to the combined string of the received trap and when coincidence - the trap is not transferred to zabbix. This is used to remove "unnecessary" traps such as Heartbeat, etc.
• aliases file, containing values of type IP - hostname. Used in cases where the traps are coming from different IP in case of multiple NICs, or where the traps come from multiple physical devices, but "logically" it would be better to keep them in one place.
• and, most interesting part, a request is made to the zabbix database to find the right item for which the trap came to (the script finds the trap key oid which is passed by value SNMPv2-MIB:: snmpTrapOID.0). Moreover, search is done by the key, and you can create a zabbix item for the different traps, and the key may be "common" for a several traps. Example: if there is zabbix host TrapTest, and for this host the item is created (such as Zabbix Trapper) with the value key = .1.3.6.1, and the second item with the value key = .1.3.6.1.4.1, when you get the trap which is decoded to 1.3.6.1.4.1.5.6 it diverted to the second item, and when the trap is decoded to .1.3.6.1.4 it is being diverted to the first item.
Disadvantages of this script:
• for each received trap a 2 DB queryis performed, which, in the presence of a large number of hosts and a lot of traps can increase the load to the database. (Although the correct use of exclude file helps to decrease load)
• for each received trap a separate process handler is started, which may also affect the performance of the machine (but that is a common problem not only for my solution, but to the implementation of the trap handling from the net-snmp, and to solve this issue, developers of zabbix must insert into their product native support for traps instead of net-snmp snmptrapd)
How to configure:
Let me show instructions on how to configure and make all this to work on the example of CentOS 5.4 and versions of zabbix 1.8 and net-snmp-5.3.2.2-7.el5
Install the perl script in any directory (I use /usr/local/sbin/), extracted from the archive.
To start you need to have a following perl modules installed: DBI and DBD::Mysql.
For my system they are available in RPM packages perl-DBI-1.52-2.el5 iperl-DBD-MySQL-3.0007-2.el5
Edit the perl script to suit your needs:
• For beginning, I recommend to enable logging
Code:
my $DEBUG = 1;
Code:
my $DEBUGFILE = "/tmp/zabbix_snmptrapdebug_new.log";
Code:
my $ZABBIX_SERVER = ""; my $ZABBIX_PORT = 10051;
Code:
my $WILDCARD_HOST = “TRAP_HOST”
) Code:
my $SEND_ALL_TO_WILDCARD = 0
Code:
my $ZABBIX_ITEM = "snmptraps";
Code:
my $ZABBIX_SENDER = "/usr/local/bin/zabbix_sender";
Code:
my $ZABBIX_DB_HOST = ""; my $ZABBIX_DB_NAME = "zabbix"; my $ZABBIX_DB_USER = "zabbix"; my $ZABBIX_DB_PASS = "";
Code:
my $TRAPEXCLUDEFILE = "/usr/local/sbin/traps_to_exclude";
Code:
my $ALIASFILE = "/usr/local/sbin/aliasfile";
Code:
my $TRIMLONGVALUES = 1;
Code:
# cat /etc/snmp/snmptrapd.conf disableAuthorization yes traphandle default /usr/local/sbin/zabbix_snmptraphandler.pl
And of course the script must have execute permissions set.
Next, we need the file describing the device SNMP, the MIB file. Which contains the actual description of how and what will be accepted by SNMP. The fact that the device sends all the information (oid) in the numeric form .1.3.6.1, etc., which is not readable and understandable by human. And the MIB file contains a description of meaning of this oid.
For standard devices the MIB files can be found on the Internet, and for your specific ones – contact the equipment supplier or look on the Internet (a very nice resource - http://www.oidview.com/mibs/detail.html).
By default (at least in my case) MIB files in the system are stored in /usr/share/snmp/mibs
Copy your file to this directory.
Edit the file /etc/snmp/snmp.conf and add there a line
Code:
mibs +MIB_MODULE_NAME
What is located to left side before DEFINITIONS, this is the right MIB module name.
(I'm doing like this - and it is working for me. Certainly you can do another way. See the man for snmp.conf)
Restart snmptrapd service:
Code:
# service snmptrad restart
Then adjust zabbix.
Create a host with the name as specified in our script (zabbix_snmptraphandler.pl) $WILDCARD_HOST. It is used when the trap is coming from the device which is not yet added to the zabbix. Add to that host item with type Zabbix trapper and the type of received information “Text”, with Key as specified in our script variable $ZABBIX_ITEM. This is the default name, which will received or traps if you have not set up the personal trap items for that host.
Now create hosts for your equipment, and add to them an item with type zabbix trapper and the type of received information “Text”, with Key as specified in our script variable $ZABBIX_ITEM. If we know beforehand the list of those traps that can come from some specific equipment, than create personal item with type zabbix trapper but in the Key we must specify numeric representation of a trap. To obtain such representation, run the command with a trap that we expect. For example:
Code:
# snmptranslate-On CPQHLTH-MIB::cpqHe4FltTolPowerSupplyOk
This is a numeric representation of trap OID.
Now, if, in the created Item’s key we define .1.3.6.1.4.1.232.0.6048, then all the traps of this type/OID will be sent to this item. If we specify in the Key .1.3.6.1.4.1.232.0 then all traps that are under this OID (a list can be found by clicking snmptranslate-Tp .1.3.6.1.4.1.232.0) will be sent to this Item . Thus, we can create a "common" Items for each type of received traps.
To make our new item working properly with the triggers we have to do following. For one previously defined item we are configuring same number of the triggers as the real SNMP traps can be send for (we have to look into the MIB file). For each trigger we put expression like:
Code:
({Template_Test_SNMPTraps:.1.3.6.1.4.1.232.1.str(Raised-alarm_Number_15)}=1 & {TRIGGER.VALUE}=0) | ({Template_Test_SNMPTraps:.1.3.6.1.4.1.232.1.str(Cleared-alarm_Number_15)}#1 & {TRIGGER.VALUE}=1)
For creating a lot of triggers I am using some simple scripts generating the xml files from MIBs for importing them into zabbix. But, as MIB files are not standard for trap descriptions, I can not share these scripts.
Comment