cannot ping switch only from zabbix server

[fleon]

Hello guys i am having a very weird issue. I am using zabbix to monitor our cisco routers and switches over SNMP. I have pings setup from zabbix and they work fine, but sometimes, when the wan frame relay links go down and come back up, the router goes up and pings are fine, but the switch "stays down", and in reality i can't even ping the switch from the OS (debian 7), until i reboot the switch. The weird thing is that i can ping the switch from my windows machine just fine, and my machine is on the same subnet as the zabbix linux server.

These are cisco 2950 switches and my understanding is that they only do layer 2 switching and in fact i can see that there are no ACLs interfering with the ping or anything.

So how come the switch is selectively blocking ping (and snmp and everything) to just the zabbix server? Is it somehow detecting too much snmp traffic requests coming from zabbix and blocking all requests? How come a reboot fixes it? This behaviour is not only on one switch, but practically everyone we have, but happens randomly.

Does anyone have an idea? I cannot keep rebooting the switch, especially since it is difficult to do it in office hours.

Traceroute to the router:
Windows machine:
1 1 ms <1 ms <1 ms 192.168.3.1
2 <1 ms <1 ms <1 ms 192.168.10.3
3 3 ms <1 ms <1 ms 192.168.10.5
4 12 ms 11 ms 12 ms 192.168.121.10

Linux server:
1 192.168.3.1 (192.168.3.1) 0.426 ms 0.886 ms 1.103 ms
2 192.168.10.3 (192.168.10.3) 1.543 ms 1.876 ms 2.190 ms
3 192.168.10.5 (192.168.10.5) 2.479 ms 1.159 ms 1.150 ms
4 10.1.1.30 (10.1.1.30) 12.314 ms * *

10.1.1.30 is the router's WAN ip.

Traceroute to the switch:
Windows machine:
1 <1 ms <1 ms <1 ms 192.168.3.1
2 1 ms <1 ms 1 ms 192.168.10.3
3 2 ms <1 ms <1 ms 192.168.10.5
4 12 ms 11 ms 11 ms 10.1.1.30
5 20 ms 12 ms 13 ms 192.168.121.3

Linux server:
1 192.168.3.1 (192.168.3.1) 0.404 ms 0.858 ms 1.075 ms
2 192.168.10.3 (192.168.10.3) 1.533 ms 2.046 ms 2.784 ms
3 192.168.10.5 (192.168.10.5) 3.236 ms 1.113 ms 1.106 ms
4 10.1.1.30 (10.1.1.30) 11.950 ms 12.354 ms 12.698 ms
5 * * *
[...]
30 * * *

[aib]

something suspicious with your LAN configuration.
I cannot understand why you have two 192.168.*.* networks on both side of your router.

Do you mind to show Network Diagram (how devices connected to each others)?
Do you mind to show network configuration on Windows machine and on Linux machine? (IP/ subnet/ default gateway)

[fleon]

we have several branches in our company. Main branch has several subnets, 192.168.3.x, 100.x, 1.x, 4.x, 5.x, 6.x,

Each branch has a 192.168.x.0/24 subnet as well.

So we have a main router where all 20 branches connect to by frame relay.

My windows machine is 192.168.3.57 , 255.255.255.128, gateway 192.168.3.1.
The zabbix linux server is 192.168.3.11, 255.255.255.128, gateway 192.168.3.1

I attach the network diagram. However my question was regarding the switch itself, not our network configuration. It doesn't make sense we have a configuration issue. I rebooted the switch at lunch and as you see the network diagram doesn't have any errors.

The traces i pasted show how to reach the switches, which are located at 192.168.x.3, while the routers are on 192.168.x.10

[zabbix_surfer]

fleon I faced the same issue on my zabbix server after rebooting. I fixed it by flushing the arp table on my switch. My zabbix lives on VM, so that after the reboot it changed its mac address and the switch wasn't anymore able to find it.