snmptraps in docker

[rajsolanki]

so after struggling with https://www.zabbix.com/forum/zabbix-...er-on-rhel-7-0 for over 3 weeks , i decided to install this in container environment. Got it up and running as per https://www.zabbix.com/documentation...ion/containers. I have 5 containers.

zabbix-agent
zabbix-web-nginx-pgsql
zabbix-server-pgsq
zabbix-snmptraps
postgres-server

my zabbix server is green in UI added snmp interface in zabbix UI using localhost , public ip and private ip of containers

docker ps shows port 0.0.0.0:162->162/udp, 1162/udp for snmp trap container.

i sent snmptrap to public ip on port 162 and i dont see anything in snmptrap log.

ps -ef inside container i see snp trapper is up

163 zabbix 0:02 /usr/sbin/zabbix_server: snmp trapper [processed data in 0.000023 sec, idle 1 sec]

i dont see snmp trapper log in zabbix container and snmp container. I have looked at github and forums but couldnt find anything. so am i missing something ?

[tim.mooney]

You don't mention snmptrapd, so forgive me but I have to ask: snmptrapd is configured to run the trap receiving script, and is listening on the correct internal port, right?

I'm specifically talking about steps 1 through 3 of the "Workflow" section of https://www.zabbix.com/documentation...types/snmptrap

If the SNMP trap log file doesn't exist or is empty, that's a clear indication that the problem is happening in the steps involved in receiving or processing the trap. Both receiving and processing are configured via snmptrapd and one of two possible scripts that are run to write the relevant information to the intermediate file. Only once it's in that file does Zabbix actually get involved.

I'm sorry you're having so much trouble getting this working. I'm still puzzled about what the original issue was, when you were trying this on RHEL 7. I'm on RHEL 7 too, but not at 5.0.x yet. I can tell you that when I set up snmp trap receiving under 3.2.x, it was working within an hour or two. It has continued to work as I've upgraded Zabbix to 4.4.x. I'm cautious about ".0" versions, so I don't plan on upgrading my Zabbix environment until maybe 5.0.3 or 5.0.4, so hopefully any early issues are worked out. One of the first things I plan on testing with whatever 5.0.x version I upgrade to is adding another snmp trap interface, just to see if I run into the same issues you did.

[rajsolanki]

Thanks for replying. ** This is install is with docker now ** . So all i m doing is starting docker images and trying to send snmp trap.

this is what i have done for docker. I have dummied password here so dont worry .

1) docker run --name postgres-server -t \
-e POSTGRES_USER="zabbix" \
-e POSTGRES_PASSWORD="dummied" \
-e POSTGRES_DB="zabbix" \
--restart unless-stopped \
-d postgres:latest

2) docker run --name zabbix-snmptraps -t \
-v /zbx_instance/snmptraps:/var/lib/zabbix/snmptraps:rw \
-v /var/lib/zabbix/mibs:/usr/share/snmp/mibs:ro \
-p 162:162/udp \
--restart unless-stopped \
-d zabbix/zabbix-snmptraps:latest

3) docker run --name zabbix-server-pgsql -t \
-e DB_SERVER_HOST="postgres-server" \
-e POSTGRES_USER="zabbix" \
-e POSTGRES_PASSWORD="dummied" \
-e POSTGRES_DB="zabbix" \
-e ZBX_ENABLE_SNMP_TRAPS="true" \
--link postgres-serverostgres \
-p 10051:10051 \
--volumes-from zabbix-snmptraps \
--restart unless-stopped \
-d zabbix/zabbix-server-pgsql:latest

4) docker run --name zabbix-web-nginx-pgsql -t \
-e DB_SERVER_HOST="postgres-server" \
-e POSTGRES_USER="zabbix" \
-e POSTGRES_PASSWORD="dummied" \
-e POSTGRES_DB="zabbix" \
--link postgres-serverostgres \
--link zabbix-server-pgsql:zabbix-server \
-p 80:8080 \
--restart unless-stopped \
-d zabbix/zabbix-web-nginx-pgsql:latest

5) docker run --name zabbix-agent --link zabbix-server-pgsql:zabbix-server -d zabbix/zabbix-agent:latest

docker ps shows everything is up and running.

About in RHEL 7 (not docker) i can not get this working and i have no idea why. I even enabled log level to trace nothing in logs. I also did install this on new server using configure and make process and i fail exactly at same place. So i just can not figure out what the issue is. Is there any way to add SNMP interface directly in SQL table ?

[rajsolanki]

and when you say i m using right internal port what do you mean ?

[rajsolanki]

i just checked snmp trap container log.

MIB search path: /usr/share/snmp/mibs/:/var/lib/zabbix/mibs
Cannot find module (SNMPv2-MIB): At line 0 in (none)
Cannot find module (IF-MIB): At line 0 in (none)
Cannot find module (IP-MIB): At line 0 in (none)
Cannot find module (TCP-MIB): At line 0 in (none)
Cannot find module (UDP-MIB): At line 0 in (none)
Cannot find module (HOST-RESOURCES-MIB): At line 0 in (none)
Cannot find module (NOTIFICATION-LOG-MIB): At line 0 in (none)
Cannot find module (DISMAN-EVENT-MIB): At line 0 in (none)
Cannot find module (DISMAN-SCHEDULE-MIB): At line 0 in (none)
Cannot find module (UCD-SNMP-MIB): At line 0 in (none)
Cannot find module (UCD-DEMO-MIB): At line 0 in (none)
Cannot find module (SNMP-TARGET-MIB): At line 0 in (none)
Cannot find module (NET-SNMP-AGENT-MIB): At line 0 in (none)
Cannot find module (HOST-RESOURCES-TYPES): At line 0 in (none)
Cannot find module (SNMP-FRAMEWORK-MIB): At line 0 in (none)
Cannot find module (SNMP-MPD-MIB): At line 0 in (none)
Cannot find module (SNMP-USER-BASED-SM-MIB): At line 0 in (none)
Cannot find module (SNMP-VIEW-BASED-ACM-MIB): At line 0 in (none)
Cannot find module (SNMP-COMMUNITY-MIB): At line 0 in (none)
Cannot find module (IPV6-ICMP-MIB): At line 0 in (none)
Cannot find module (IPV6-MIB): At line 0 in (none)
Cannot find module (IPV6-TCP-MIB): At line 0 in (none)
Cannot find module (IPV6-UDP-MIB): At line 0 in (none)
Cannot find module (IP-FORWARD-MIB): At line 0 in (none)
Cannot find module (NET-SNMP-PASS-MIB): At line 0 in (none)
Cannot find module (NET-SNMP-EXTEND-MIB): At line 0 in (none)
Cannot find module (UCD-DLMOD-MIB): At line 0 in (none)
Cannot find module (SNMP-NOTIFICATION-MIB): At line 0 in (none)
Cannot find module (SNMPv2-TM): At line 0 in (none)
Cannot find module (NET-SNMP-VACM-MIB): At line 0 in (none)
NET-SNMP version 5.8
2020-06-02 02:43:57,496 INFO spawned: 'snmptrapfmt' with pid 9
2020-06-02 02:43:57,496 INFO spawned: 'snmptrapfmt' with pid 9
Cannot open log file '/var/tmp/snmptrapfmt.trc' [File exists]
2020-06-02 02:43:57,502 INFO exited: snmptrapfmt (exit status 1; not expected)
2020-06-02 02:43:57,502 INFO exited: snmptrapfmt (exit status 1; not expected)
2020-06-02 02:43:59,505 INFO spawned: 'snmptrapfmt' with pid 10
2020-06-02 02:43:59,505 INFO spawned: 'snmptrapfmt' with pid 10
2020-06-02 02:43:59,505 INFO success: snmptrapd entered RUNNING state, process has stayed up for > than 3 seconds (startsecs)
2020-06-02 02:43:59,505 INFO success: snmptrapd entered RUNNING state, process has stayed up for > than 3 seconds (startsecs)
Cannot open log file '/var/tmp/snmptrapfmt.trc' [File exists]
2020-06-02 02:43:59,511 INFO exited: snmptrapfmt (exit status 1; not expected)
2020-06-02 02:43:59,511 INFO exited: snmptrapfmt (exit status 1; not expected)
2020-06-02 02:44:02,515 INFO spawned: 'snmptrapfmt' with pid 11
2020-06-02 02:44:02,515 INFO spawned: 'snmptrapfmt' with pid 11
Cannot open log file '/var/tmp/snmptrapfmt.trc' [File exists]
2020-06-02 02:44:02,521 INFO exited: snmptrapfmt (exit status 1; not expected)
2020-06-02 02:44:02,521 INFO exited: snmptrapfmt (exit status 1; not expected)
2020-06-02 02:44:02,521 INFO gave up: snmptrapfmt entered FATAL state, too many start retries too quickly
2020-06-02 02:44:02,521 INFO gave up: snmptrapfmt entered FATAL state, too many start retries too quickly


and that file has

cat /var/tmp/snmptrapfmt.trc
20200602.012433 (snmptrapfmt-7): Cannot append to logfile '/var/lib/zabbix/snmptraps/snmptraps.log' [Permission denied]

so how to fix this ?

[tim.mooney]

and when you say i m using right internal port what do you mean ?

I mean that in your first post in this thread, you said:

docker ps shows port 0.0.0.0:162->162/udp, 1162/udp for snmp trap container.

Which isn't super clear, but implies that external port 162/udp might be mapped to port 1162/udp inside the container. I wasn't certain how to read that, but wanted to be certain that the port mapping between the outside world and the inside of the container had your snmptrapd listening on the correct port inside the container.

In your post #3 in this thread the docker run you show makes it appear that the ports are mapped internally and externally to the same port, so perhaps the first 1162/udp was just a typo?

[rajsolanki]

yes port are mapped with public ip. The thing is snmp trap container it self isnt starting as its missing things. I have opened a github issue. lets see if some one responds. BTW i went back to 4.4 and i have it all working. So i dont even know if v5 is for me lol

[damasu]

Sorry to jump in here, but I am lost on how to properly test that a docker-compose setup with zabbix-snmpstraps is correctly setup. - version 4.4 as an example.

1. Considering that with docker-compose we are using just 162->1162 on the zabbix-snmptraps, what exactly is our SNMP interface?
2. How do we actually use snmptrap to make a test, for instance, from the host or another docker image?

################################################
# SNMP: trapping agent
################################################
zabbix-snmptraps:
image: zabbix/zabbix-snmptraps:ubuntu-4.4-latest
container_name: zabbix-snmptraps
restart: always
ports:
- "162:1162/udp"
volumes:
- /data/zabbix/snmptraps:/var/lib/zabbix/snmptraps:rw
​​​​​​

################################################
# Zabbix: Server
################################################
zabbix-server:
build:
context: .
dockerfile: ./docker/zabbix-server/Dockerfile
container_name: zabbix-server
restart: always
env_file:
- .env_srv
- .env_db_pgsql
ports:
- "10051:10051"
depends_on:
- postgres
- zabbix-snmptraps
volumes:
- /data/zabbix/alertscripts/:/usr/lib/zabbix/alertscripts/
- /data/zabbix/externalscripts/:/usr/lib/zabbix/externalscripts/
- /data/zabbix/snmptraps:/var/lib/zabbix/snmptraps:ro

ZBX_ENABLE_SNMP_TRAPS=true

[damasu]

So i think some communication is occurring but the output looks wrong.

tail -f /data/zabbix/snmptraps/snmptraps.log

20200603.210033 ZBXTRAP 172.20.0.1 6 33 (null) iso.3.6.1.6.3.1.1.5.4 eth0 iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
20200603.210037 ZBXTRAP 172.20.0.1 6 33 (null) iso.3.6.1.6.3.1.1.5.4 eth0 iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
20200603.210038 ZBXTRAP 172.20.0.1 6 33 (null) iso.3.6.1.6.3.1.1.5.4 eth0 iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
20200603.210038 ZBXTRAP 172.20.0.1 6 33 (null) iso.3.6.1.6.3.1.1.5.4 eth0 iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
20200603.210120 ZBXTRAP 172.20.0.1 6 33 (null) iso.3.6.1.6.3.1.1.5.4 lo iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
20200603.210124 ZBXTRAP 172.20.0.1 6 33 (null) iso.3.6.1.6.3.1.1.5.4 lo iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
20200603.210132 ZBXTRAP 172.20.0.1 6 33 (null) iso.3.6.1.6.3.1.1.5.4 eth0 iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
20200603.210141 ZBXTRAP 172.20.0.1 6 33 (null) iso.3.6.1.6.3.1.1.5.4 ens3 iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
20200603.211721 ZBXTRAP 172.20.0.9 6 33 (null) iso.3.6.1.6.3.1.1.5.4 eth0 iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
20200603.211749 ZBXTRAP 172.20.0.9 6 33 (null) iso.3.6.1.6.3.1.1.5.4 eth0@if268 iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
20200603.212029 ZBXTRAP 172.20.0.9 6 33 (null) iso.3.6.1.6.3.1.1.5.4 eth0@if268 iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
20200603.212031 ZBXTRAP 172.20.0.9 6 33 (null) iso.3.6.1.6.3.1.1.5.4 eth0@if268 iso.3.6.1.6.3.18.1.4.0 public iso.3.6.1.6.3.1.1.4.3.0 iso.3.6.1.6.3.1.1.5.4
​​​​​​

sent from a dummy alpine container with snmpd enabled

snmptrap -v 1 -c public 172.20.0.7:1162 '.1.3.6.1.6.3.1.1.5.4' '0.0.0.0' 6 33 '55' .1.3.6.1.6.3.1.1.5.4 s "eth0"

sent from host

snmptrap -v 1 -c public 127.0.0.1 '.1.3.6.1.6.3.1.1.5.4' '0.0.0.0' 6 33 '55' .1.3.6.1.6.3.1.1.5.4 s "eth0"

Any ideas what is going wrong here?

Again, i am using the zabbix-snmptraps ubuntu 4.4-latest image

like rajsolanki i am seeing a lot of errors in the stdout from the docker container (snmptraps) related to MIB

[damasu]

so i believe i found the problem. testing now; however, it is directly related to the MIB OID errors in the container. I had to build on top of the orignial Dockerfile

FROM zabbix/zabbix-snmptraps:ubuntu-4.4-latest

LABEL maintainer="damasu"

USER root

RUN apt-get update && apt-get -y install && \
apt-get install nano bash telnet snmp-mibs-downloader -y && \
download-mibs && \
apt-get clean && rm -rf /var/lib/apt/lists/*

basically snmp-mibs-downloader and download-mibs was required.

[rajsolanki]

i m surprised you did not get permission error on snmptrap log coz for me i get /var/lib/zabbix/snmptraps/snmptraps.log'. Such a simple thing doesnt work for me

[damasu]

i m surprised you did not get permission error on snmptrap log coz for me i get /var/lib/zabbix/snmptraps/snmptraps.log'. Such a simple thing doesnt work for me

I have the following tree format

├── alertscripts
│ ├── requirements.txt
│ └── zabbix2logstash.py
├── externalscripts
│ ├── elasticsearch_api.py
│ └── elasticsearch_api_zabbix_keys.py
├── mibs
└── snmptraps
└── snmptraps.log

with drwxr-xr-x. 2 1997 root 27 Jun 3 19:49 snmptraps as an example. no issues with permissions.

[rajsolanki]

where did you get that tree from ?

[damasu]

where did you get that tree from ?

you can see from the docker-compose info i gave above, i am using a volume mounted from the host. that information is from that mount.

[rajsolanki]

well in my case its not like what you have


in my case when i attach to my container and go here

/var/lib/zabbix $ ls -ltr
total 0
drwxr-xr-x 2 root root 6 Jun 4 18:34 snmptraps
drwxrwxr-x 2 zabbix root 6 Jun 4 19:21 mibs

snmptrap is owned by root.



i m using compose yml which is on zabbix git hub https://github.com/zabbix/zabbix-doc...ql_latest.yaml