Crucial information:
Production environment:
Host OS: ubuntu-26.04-live-server-amd64
Zabbix version: 7.4.13
Database: MySQL
Webserver: Nginx
Template used: Windows by SNMP
Template Vendor and version: Zabbix, 7.4-4
Test environment:
Host OS: Zabbix appliance (Alma Linux 8)
Zabbix version: 7.4.13
Database: MySQL
Webserver: Nginx
Template used: Windows by SNMP
Template Vendor and version: Zabbix, 7.2-2
Small introduction: In the past I had an small Zabbix setup. Overwhelmed by all the options, I choose to switch to NagiosCSP. It worked for us, easier to setup and free for 50 devices. Unfortunately, because of changes in their policy, the CSP license is revoked. With my small experience with Zabbix I choose to come back. I've made myself an test environment based on the Zabbix appliance. Added an host, with help of AI created a specific service readout with a trigger. It was quite doable for me, and it no longer felt like a raging river. By testing things at my leisure I was getting somewhere. But, for the appliance I read that it is not the best option for production purposes. So, pulling the Ubuntu ISO to our cluster, attached it to the Nagios VM and formatted the whole server. LET'S START! I felt confident as my test-enviroment did what i wanted.
Server installed etcetc. Added a host, just like my test environment. Same host, same template, same SNMP settings etc. Wait, WHAT? Where I had 135 items and 15 triggers for the host in the test environment, I now have 203 items and 90 triggers for the same host in my 'production'-environment! In my search for the difference, I found out there where more network interfaces added. In test: 1 interface, production around 7 I guess? As I'm using the same template etc, I tried to investigate where the difference is made.
With help of AI (and running around a lot in the same circles, TNX AI!) it seems that the appliance (test environment) receives the same data for the network interfaces discovery. But, in production it seems that the interfaces that would normally be filtered out by the macro's (which apparently works in test) are now still added. While comparing data it looks like the test environment for example receives the netwerk interface names as QoS, teredo etc. In production, it looks like the interface names are moved to the interface description, where a different macro is applied and therefore the interfaces aren't filtered out. But, this is a guess...
Some information from the database to show the found interfaces:
Production:
Test:
When running an SNMP walk to the same host, there are also differences to see: On the production server the response is in HEX, on the testserver i receive a response in text. A small part of the output:
Production:
Test:
Can someone give me a helping hand in achieving the same results in both enviroments?
Production environment:
Host OS: ubuntu-26.04-live-server-amd64
Zabbix version: 7.4.13
Database: MySQL
Webserver: Nginx
Template used: Windows by SNMP
Template Vendor and version: Zabbix, 7.4-4
Test environment:
Host OS: Zabbix appliance (Alma Linux 8)
Zabbix version: 7.4.13
Database: MySQL
Webserver: Nginx
Template used: Windows by SNMP
Template Vendor and version: Zabbix, 7.2-2
Small introduction: In the past I had an small Zabbix setup. Overwhelmed by all the options, I choose to switch to NagiosCSP. It worked for us, easier to setup and free for 50 devices. Unfortunately, because of changes in their policy, the CSP license is revoked. With my small experience with Zabbix I choose to come back. I've made myself an test environment based on the Zabbix appliance. Added an host, with help of AI created a specific service readout with a trigger. It was quite doable for me, and it no longer felt like a raging river. By testing things at my leisure I was getting somewhere. But, for the appliance I read that it is not the best option for production purposes. So, pulling the Ubuntu ISO to our cluster, attached it to the Nagios VM and formatted the whole server. LET'S START! I felt confident as my test-enviroment did what i wanted.
Server installed etcetc. Added a host, just like my test environment. Same host, same template, same SNMP settings etc. Wait, WHAT? Where I had 135 items and 15 triggers for the host in the test environment, I now have 203 items and 90 triggers for the same host in my 'production'-environment! In my search for the difference, I found out there where more network interfaces added. In test: 1 interface, production around 7 I guess? As I'm using the same template etc, I tried to investigate where the difference is made.
With help of AI (and running around a lot in the same circles, TNX AI!) it seems that the appliance (test environment) receives the same data for the network interfaces discovery. But, in production it seems that the interfaces that would normally be filtered out by the macro's (which apparently works in test) are now still added. While comparing data it looks like the test environment for example receives the netwerk interface names as QoS, teredo etc. In production, it looks like the interface names are moved to the interface description, where a different macro is applied and therefore the interfaces aren't filtered out. But, this is a guess...
Some information from the database to show the found interfaces:
Production:
Code:
mysql> SELECT DISTINCT
-> SUBSTRING_INDEX(
-> SUBSTRING_INDEX(i.name,'(',-1),
-> ')',1
-> ) AS interface_name
-> FROM items i
-> JOIN hosts h ON h.hostid = i.hostid
-> WHERE h.host = 'SVR-MAC-APP03'
-> AND i.flags = 4
-> AND i.key_ LIKE 'net.if.in[%'
-> ORDER BY interface_name;
+------------------------------------------------------------------+
| interface_name |
+------------------------------------------------------------------+
| Ethernet0 |
| Ethernet0-QoS Packet Scheduler-0000 |
| Ethernet0-WFP 802.3 MAC Layer LightWeight Filter-0000 |
| Ethernet0-WFP Native MAC Layer LightWeight Filter-0000 |
| Local Area Connection* 1 |
| Local Area Connection* 2 |
| Local Area Connection* 3 |
| Local Area Connection* 4 |
| Local Area Connection* 5 |
| Local Area Connection* 6 |
| Local Area Connection* 7 |
| Local Area Connection* 7-QoS Packet Scheduler-0000 |
| Local Area Connection* 7-WFP Native MAC Layer LightWeight Filter |
| Local Area Connection* 8 |
| Local Area Connection* 8-QoS Packet Scheduler-0000 |
| Local Area Connection* 8-WFP Native MAC Layer LightWeight Filter |
| Local Area Connection* 9 |
| Local Area Connection* 9-QoS Packet Scheduler-0000 |
| Local Area Connection* 9-WFP Native MAC Layer LightWeight Filter |
| Loopback Pseudo-Interface 1 |
+------------------------------------------------------------------+
20 rows in set (0.00 sec)
Code:
mysql> SELECT DISTINCT
-> SUBSTRING_INDEX(
-> SUBSTRING_INDEX(i.name,'(',-1),
-> ')',1
-> ) AS interface_name
-> FROM items i
-> JOIN hosts h ON h.hostid = i.hostid
-> WHERE h.host = 'SVR-MAC-APP03'
-> AND i.flags = 4
-> AND i.key_ LIKE 'net.if.in[%'
-> ORDER BY interface_name;
+----------------+
| interface_name |
+----------------+
| Ethernet0 |
+----------------+
1 row in set (0.00 sec)
Production:
Code:
snmpwalk -v2c -c thisissecret:) hostip 1.3.6.1.2.1.2.2.1.2 iso.3.6.1.2.1.2.2.1.2.1 = Hex-STRING: 53 6F 66 74 77 61 72 65 20 4C 6F 6F 70 62 61 63 6B 20 49 6E 74 65 72 66 61 63 65 20 31 00 iso.3.6.1.2.1.2.2.1.2.2 = Hex-STRING: 57 41 4E 20 4D 69 6E 69 70 6F 72 74 20 28 4E 65 74 77 6F 72 6B 20 4D 6F 6E 69 74 6F 72 29 00
Code:
snmpwalk -v2c -c thisissecret:) hostip 1.3.6.1.2.1.2.2.1.2 IF-MIB::ifDescr.1 = STRING: Software Loopback Interface 1. IF-MIB::ifDescr.2 = STRING: WAN Miniport (Network Monitor).
Comment