Trouble monitoring SNMP items - Parameter not supported

[nottestuser]

I'm having difficulty monitoring items with SNMP. I have a Zabbix 1.6.2 install from Fedora's development branch and I'm trying to monitor some APC UPSs.

When I configure the SNMP items according to

http://www.zabbix.com/wiki/doku.php?id=contrib:snmp

I get this in the logs:

 24742:20090304:132821 In int_in_list(list:,value:10052)
 24742:20090304:132821 End int_in_list(ret:FAIL)
 24742:20090304:132821 In get_value(key:upsAdvOutputLoad)
 24742:20090304:132821 In get_value_snmp(key:upsAdvOutputLoad, oid:.1.3.6.1.4.1.318.1.1.1.4.2.3.0)
 24742:20090304:132821 Standard processing
 24742:20090304:132821 In snmp_normalize(oid:.1.3.6.1.4.1.318.1.1.1.4.2.3.0)
 24742:20090304:132821 End of snmp_normalize(result:.1.3.6.1.4.1.318.1.1.1.4.2.3.0)
 24742:20090304:132821 In get_snmp(oid:.1.3.6.1.4.1.318.1.1.1.4.2.3.0)
 24742:20090304:132821 SNMP [[email protected]:161:161]
 24742:20090304:132826 Timeout while answering request
 24742:20090304:132827 Status send [2]
 24742:20090304:132827 Item [ups1r1-m006:upsAdvOutputLoad] error: Timeout while connecting to [10.5.83.23:161]
 24742:20090304:132827 End get_value()
 24742:20090304:132827 Parameter [upsAdvOutputLoad] is not supported by agent on host [ups1r1-m006] Old status [0]
 24742:20090304:132827 In zabbix_log()

And in the interface I see that the items are all listed as "Timeout while connecting to [10.5.83.23:161]".

I've placed the APC mib in /usr/share/snmp/mibs on the zabbix server. From the same box I can run the Nagios APC SNMP monitoring perl script, snmpwalk, and snmpget for that oid and name and get results, for instance:

$ snmpwalk -v 1 -c public -m ALL ups1r1-m006 PowerNet-MIB::upsAdvOutputLoad.0
PowerNet-MIB::upsAdvOutputLoad.0 = Gauge32: 11
$ snmpwalk -v 1 -c public -m ALL ups1r1-m006 .1.3.6.1.4.1.318.1.1.1.4.2.3.0
PowerNet-MIB::upsAdvOutputLoad.0 = Gauge32: 11

When looking at the tcpdump output for UDP traffic on port 161, I can watch the queries go out and get replies back. The query being issued by Zabbix is similar to the ones issued by snmpwalk and by the Nagios check script:

snmpwalk tcpdump (remember, this works):

13:39:40.209809 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: UDP (17), length: 77) zabbix.32849 > ups1r1-m006.snmp: [bad udp cksum badd!]  { SNMPv1 { GetNextRequest(34) R=1463898493  E:318.1.1.1.4.2.3.0 } }
13:39:40.236000 IP (tos 0x0, ttl  64, id 39525, offset 0, flags [none], proto: UDP (17), length: 86) ups1r1-m006.snmp > zabbix.32849: [no cksum]  { SNMPv1 { GetResponse(39) R=1463898493  E:318.1.1.1.4.2.4[|snmp] } }
13:39:40.236111 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: UDP (17), length: 77) zabbix.32849 > ups1r1-m006.snmp: [bad udp cksum bbdc!]  { SNMPv1 { GetRequest(34) R=1463898494  E:318.1.1.1.4.2.3.0 } }
13:39:40.263520 IP (tos 0x0, ttl  64, id 39526, offset 0, flags [none], proto: UDP (17), length: 86) ups1r1-m006.snmp > zabbix.32849: [no cksum]  { SNMPv1 { GetResponse(39) R=1463898494  E:318.1.1.1.4.2.3[|snmp] } }

zabbix tcpdump:

11:49:36.884714 IP (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: UDP (17), length: 77) int-old.flmnh.ufl.edu.47794 > ups1r1-m006.flmnh.ufl.edu.snmp: [udp sum ok]  { SNMPv1 { GetRequest(34) R=343084436  E:318.1.1.1.4.2.3.0 } }
11:49:36.906781 IP (tos 0x0, ttl  64, id 38746, offset 0, flags [none], proto: UDP (17), length: 86) ups1r1-m006.flmnh.ufl.edu.snmp > int-old.flmnh.ufl.edu.47794: [no cksum]  { SNMPv1 { GetResponse(39) R=343084436  E:318.1.1.1.4.2.3[|snmp] } }

So it's asking the right questions on the wire, getting seemingly valid responses from the device, but Zabbix's showing the item as a timeout? Any help would be appreciated.

[neha]

Hello -

I can also confirm this issue on Solaris 10. This feels like some sort of bug, except in my case, the snmp GET's are issued from the Zabbix server, but my Cisco Catalyst device doesn't return a response. I can GET the same OID from snmpget and/or my Nagios installation. Zabbix seems to have some issue with possibly the item key (Parameter not Supported)... maybe even malformed SNMP packets? I will gather more information and attempt other devices, but I can at least confirm an issue of a similar nature to yours.

Solaris 10
ZABBIX Server (daemon) v1.6.2 (16 January 2009)
Compilation time: Feb 5 2009 12:21:48

netsnmp-5.4.2.1-sol10-sparc-local

[neha]

Packet Captures for this Issue

After reviewing my configs, double-checking with a different SNMP host, and testing with snmpget, I've come across a peculiar discovery.

I'm noticing that the SNMP Get Requests that Zabbix 1.6.2 sends on the wire to the SNMP device appear to lack a proper community string.

My Setup:


I notice logs similar to this for the SNMP hosts that are timing out:

Item [switch31.prod.sjc.pno.net:ifInOctets.10125] error: Timeout while connecting to [192.168.255.21:161]

Packet Capture of a Zabbix GetRequest (notice C= null):

15:53:13.338441 IP t1-026.63835 > switch31.prod.sjc.pno.net.161: C= GetRequest(31) interfaces.ifTable.ifEntry.ifInOctets.10125
15:53:14.341509 IP t1-026.63835 > switch31.prod.sjc.pno.net.161: C= GetRequest(31) interfaces.ifTable.ifEntry.ifInOctets.10125
15:53:15.351239 IP t1-026.63835 > switch31.prod.sjc.pno.net.161: C= GetRequest(31) interfaces.ifTable.ifEntry.ifInOctets.10125
15:53:16.361033 IP t1-026.63835 > switch31.prod.sjc.pno.net.161: C= GetRequest(31) interfaces.ifTable.ifEntry.ifInOctets.10125
15:53:17.370791 IP t1-026.63835 > switch31.prod.sjc.pno.net.161: C= GetRequest(31) interfaces.ifTable.ifEntry.ifInOctets.10125
15:53:18.380549 IP t1-026.63835 > switch31.prod.sjc.pno.net.161: C= GetRequest(31) interfaces.ifTable.ifEntry.ifInOctets.10125

Packet Capture of an snmpget GetRequest, in which the same device responds with data (note the C=$communitystring):

15:55:13.612287 IP t1-026.63841 > switch31.prod.sjc.pno.net.161: C=public GetRequest(31) interfaces.ifTable.ifEntry.ifInOctets[|snmp]
15:55:13.615540 IP switch31.prod.sjc.pno.net.161 > t1-026.63841: C=public GetResponse(36) interfaces.ifTable.ifEntry.ifInOctets[|snmp]

I notice that in the OPs post, the packets are marked with 'bad udp cksum'. I think these issues may be related. In my case, Zabbix has completely left out the community string and as a result, the device will never respond.

Any ideas?

[nottestuser]

Is the community string set in the database? I seem to recall reading a post where someone was entering a community string in the interface and it wasn't being saved. Try checking in the MySQL (or other) database tables to make sure that field is what you expect?

This is a multi-homed box and I'm thinking that maybe the replies are not going to the right interface. I'm going to look at that next.

[neha]

Ahh, ok - that's a good clue. I'll check there. For right now, I'm recompiling with an updated net-snmp release. When compiling Zabbix out of the box, make complains about a 'localname' variable in 'checks_snmp.c', which is linked into the poller code. I found a couple of forum threads where the recommendation is to comment the lines our or update your ver of snmp.

I commented out the lines originally, but today I rechecked my build environment and realized that I (accidentally) built the first zabbix_server pointing to the wrong (older) installation of net-snmp...

I'm going to recompile properly and check the zabbix_server. I'll definitely take a look at the db tables as well - it's definitely not being set or issued *somewhere*...

thanks!

[neha]

So it looks like a rebuild of the zabbix_server code with an updated net-snmp (5.4) solved this issue for me...