SNMP Low Level Discovery and a Cisco 6509 switch

[ericgearhart]

It seems I've finally narrowed down some of the problems I've been having with low level discovery, and by upping the number of poller threads in zabbix_server.conf I think I finally have SNMP LLD working against about 10 Cisco devices I've added to Zabbix

The one problem that has remained through all this that I can't figure out is SNMP against a 6509 switch. Building items "the old fashioned way" and specifying specific OIDs in the items (such as ifInOctet.1 and ifOutOctet.1 for example) and low level discovery both seem to generate problems in zabbix_server.log.

Has anyone else had problems doing SNMP against Cisco 6509 switches? This is the last major hurdle in a new Zabbix deployment at work, and I think I've got the folks at work to have enough confidence in Zabbix at this point to actually deploy it in production (hint: paying customers!)

[jesmit]

I'm actually monitoring multiple 6509's in both VSS and HSRP. I've got multiple discovery rules pulling in hardware, hsrp, and interfaces.

I'm more than happy to help if you let me know more specifically what the problem is.

[ericgearhart]

Really? If you don't mind me asking, what are you running Zabbix on? What OS and what version of Zabbix? Also are you using SNMPv3 or v2?

I'm running 2.0.0rc2 on RedHat 6 and using SNMPv3 exclusively, and I can't for the life of me get SNMP to work correctly against the 6509s we have.

The discovery process seems to work fine in Zabbix and it finds a bunch of interfaces, but when the items discovered via LLD actually try to start polling, the host ends up disabled in Zabbix and the frontend simply says "Connection timed out"

You'd think this is an SNMP config problem in some way, but snmpwalks and snmpgets work perfectly fine during the entire testing process

See this thread: http://www.zabbix.com/forum/showthread.php?t=25722 for more specific symptoms... NETWORK_ERROR when I do 'Debug=4' in zabbix_server.conf is the biggest and most obvious thing in the zabbix_server.log

[ericgearhart]

Just in case anyone else stumbles across this thread, please see this ticket: https://support.zabbix.com/browse/ZBX-4901 for a description of the problem I'm experiencing... I'm convinced it's a bug in the Zabbix SNMPv3 poller code at this point

[ericgearhart]

Here's a bit of an update to this one, based on work I did over the weekend.

The original discovery rules I had in place had different SNMPv3 'auth' and 'priv' passwords. I changed the passwords to line up, and SNMPv3 based discovery seems to be working now.

Secondly, in the Zabbix source tree, specifically in src/zabbix_server/poller/checks_snmp.c, in the get_snmp function I changed the code that returns NETWORK_ERROR to return NOTSUPPORTED. I know this is a hacky fix (if the device goes down I'm going to have queue backups) but I wanted to see if I could get SNMPv3 based low level discovery working.

This is the resulting code I now have running:

               else if (STAT_TIMEOUT == status)
                {
                        SET_MSG_RESULT(value, zbx_dsprintf(NULL, "get_snmp: Timeout while connecting to [[%s]:%d]",
                                        item->interface.addr, (int)item->interface.port));
                        /* ret = NETWORK_ERROR; */
                        ret = NOTSUPPORTED;
                }

Making my auth and priv SNMPv3 passwords match, and commenting out the NETWORK_ERROR return value and replacing it with NOTSUPPORTED in the get_snmp function has SNMPv3 low level discovery and polling working flawlessly for me on two Cisco 6509 switches with close to 2,000 (!) items