SNMP Trap Message Parsing

[srupp]

Greetings all,

I have setup and have snmp trapping working correcting for several devices on my network. However, I have one vendor of devices that I cannot seem to trap correctly.

Here is a sample of the trap data:

June 8 09:20:30 zabbix snmptrapd[1041]: 10.100.5.109: Enterprise Specific Trap (AIDU-MIB::alarmTrap) Uptime: 118 days, 19:27:45.46, AIDU-MIB::genEquipCurrentAlarmCounter = INTEGER: 589246043, AIDU-MIB::genEquipCurrentAlarmRaisedTimeT = INTEGER: 1305902292, AIDU-MIB::genEquipCurrentAlarmId = INTEGER: 306, AIDU-MIB::genEquipCurrentAlarmSlotId = INTEGER: 1, AIDU-MIB::genEquipCurrentAlarmInstance = INTEGER: 1, AIDU-MIB::genEquipCurrentAlarmSeverity = INTEGER: warning(4), AIDU-MIB::genEquipCurrentAlarmIfIndex = INTEGER: 407, AIDU-MIB::genEquipCurrentAlarmModule = STRING: "IDU", AIDU-MIB::genEquipCurrentAlarmDesc = STRING: "E1/DS1 unexpected signal on port #7", AIDU-MIB::genEquipMostSevereAlarm = INTEGER: major(2), AIDU-MIB::genEquipNetworkAgentIp = IpAddress: 10.100.5.109, AIDU-MIB::genEquipCurrentAlarmState = INTEGER: cleared(0), AIDU-MIB::genEquipTrapCfgMgrCLLI = ""

I can verify that the above trap was received by Zabbix as it is recorded in the syslog. However when I check the latest data for the device that should receive the trap, the history is empty. I can confirm the device is working if I pass direct zabbix_sender commands to be processed.

I'm guessing the problem has something to do with the nature of the structure of the trap. Here is my snmptrap.sh (taken from the Zabbix book):

- - - -

# CONFIGURATION

ZABBIX_SERVER="localhost";
ZABBIX_PORT="10051";

ZABBIX_SENDER="/usr/bin/zabbix_sender";

KEY="snmptraps";
HOST="snmptraps";

# END OF CONFIGURATION

read hostname
read ip
read uptime
read oid
read address
read community
read enterprise

oid=`echo $oid|cut -f2 -d' '`
address=`echo $address|cut -f2- -d' '`
community=`echo $community|cut -f2 -d' '`
enterprise=`echo $enterprise|cut -f2 -d' '`

oid=`echo $oid|cut -f11 -d'.'`
community=`echo $community|cut -f2 -d'"'`

export HOME=/root
ZABBIXHOSTID=$(echo "select hostid,host from zabbix.hosts where ip=\"$hostname\" order by 'hostid' limit 1;" | mysql -N 2> /dev/null)
ZABBIXID=$(echo $ZABBIXHOSTID | cut -d" " -f1)
ZABBIXHOST=$(echo $ZABBIXHOSTID | cut -d" " -f2-)
[[ "$ZABBIXHOST" ]] && {
TRAPOID=$(echo $oid | cut -d: -f3)
if [ "$TRAPOID" ]; then
ZABBIXITEM=$(echo "select key_ from zabbix.items where key_=\"$TRAPOID\" and hostid=\"$ZABBIXID\";" | mysql -N 2> /dev/null)
if [ "$ZABBIXITEM" ]; then
KEY=$ZABBIXITEM
HOST=$ZABBIXHOST
elif [ "$(echo "select key_ from zabbix.items where key_='snmptraps' and hostid=\"$ZABBIXID\";" | mysql -N 2> /dev/null)" ]; then
HOST=$ZABBIXHOST
fi
elif [ "$(echo "select key_ from zabbix.items where key_='snmptraps' and hostid=\"$ZABBIXID\";" | mysql -N 2> /dev/null)" ]; then
HOST=$ZABBIXHOST
fi
}

str="$hostname $address $community $enterprise $oid"

$ZABBIX_SENDER -z $ZABBIX_SERVER -p $ZABBIX_PORT -s "$HOST" -k "$KEY" -o "$str"

- - - -

I'm unsure how to interrupt what data is being pulled from the trap and processed by the script. At first I thought it was because the trap message ended in ("") which might indicate there is no data to pass to Zabbix but even using Zabbix_sender commands with empty data I at last have a blank history entry.

Would anyone be able to give me a few hints please as to how to begin working this problem?

Thank you kindly,
Stephen

[srupp]

I resolved the issue myself.

[angelhc]

And how you did it?
Cause I've the same configuration and it doesn't run.

Thanks.

[srupp]

In my case, it was not Zabbix or the SNMPTrapd at fault, but rather something I had to learn regarding the Ceragon devices I was trapping.

The equipment I was trying to receive traps from, which I had built and configured as a single Zabbix host, was actually a virtual IP shared between two physical devices. After a review of raw snmptrap data and the syslog I finally noted the different IPs. (In my attempt to troubleshoot this, I got really crazy with modifying the Zabbix snmptrap.sh to dump the raw snmptraps and the groomed data into flat files so I could compare output.)

In summary, the virtual IP was the address I was doing SNMP queries to and therefore I never considered I was missing a piece of the puzzle. I learned if I wanted to receive traps, I had to build two more devices with the physical IPs for Zabbix to associate the incoming trap info with a known host (standard practice of course).

However, once I build the two physical devices and added the snmptrap item to them, I was able to verify from "latest data" that everything was working. I built my triggers and actions from there and everything is working. My new battle is how to control the rate at which my actions execute.

That is to say, I have been directed to watch so many traps that one event will generate 40 emails. I need to figure out a way to rate limit all the emails/SMS messages this dumps on my mobile and mailbox.

Stephen

[ghoz]

defining depedencies may help, but not solve the problem...

if a device errors means all the subdevices behind it will error too,
you could make the subdevices depend on the first...

however the way dependencies work if you receive get the triggers /traps in the wrong order you'll still get multiple alerts...

if you realy want to rate limit , I think you have to create your own script to send that many text messages before dropping the alerts on the floor.
You'd better test it a lot before putting in production ...

[zalex_ua]

That is to say, I have been directed to watch so many traps that one event will generate 40 emails. I need to figure out a way to rate limit all the emails/SMS messages this dumps on my mobile and mailbox.

Stephen

You can vote for ZBXNEXT-39