active check got (ZAB_TCP_READ) time out

[mehdiaghayari]

I used zabbix appliance docker image (centos created 2018-11-26) on docker-ce (17.06.0) with below command.

docker run --restart=always --name zabbix -h zabbix -e ZBX_SERVER_NAME="zabbix" -p 80:80 -p 10051:10051 -d zabbix/zabbix-appliance:centos-trunk

also I used zabbix-agent (rpm version) on centos 6 and 7, after restart zabbix-agent, I got bellow error.
active check configuration update from [zabbix_server_IP:10051] started to fail (ZAB_TCP_READ) time out()

also, I set Server and ServerActive and ,... on zabbix_agentd.conf and there is no network or firewall problem because below telnets are OK:
telnet zabbix_server_ip:10051
telnet zabbix_agent_ip:10050

but, zabbix agent (active) items not working, for example log[] and logrt[]

please assist me.

[mehdiaghayari]

Additional information:

zabbix_agentd.conf
PidFile=/var/run/zabbix/zabbix_agentd.pid
LogFile=/var/log/zabbix/zabbix_agentd.log
LogFileSize=0
DebugLevel=3
EnableRemoteCommands=0
LogRemoteCommands=0
Server=x.x.x.x
ServerActive=x.x.x.x
Hostname=zabbix
Timeout=20
Include=/etc/zabbix/zabbix_agentd.d/*.conf

zabbix_server.conf
LogType=console
DBHost=localhost
DBName=zabbix
DBUser=zabbix
DBPassword=********
DBPort=3306
javaGateway=localhost
javaGatewayPort=10052
StartJavaPollers=5
Timeout=13
AlertScriptsPath=/usr/lib/zabbix/alertscripts
ExternalScripts=/usr/lib/zabbix/externalscripts

Also there is no same issue on zabbix server (rpm package) that installed on centos 7.

Is there any idea?

[mehdiaghayari]

I increase DebugLevel to 4 on agent and there is bellow logs

18587:20181225:152349.105 In zbx_tls_init_child()
18587:20181225:152349.108 OpenSSL library (version OpenSSL 1.0.1e-fips 11 Feb 2013) initialized
18587:20181225:152349.108 End of zbx_tls_init_child()
18587:20181225:152349.108 In init_active_metrics()
18587:20181225:152349.108 buffer: first allocation for 100 elements
18587:20181225:152349.108 End of init_active_metrics()
18587:20181225:152349.108 In send_buffer() host:'172.17.0.2' port:10051 entries:0/100
18587:20181225:152349.108 End of send_buffer():SUCCEED
18587:20181225:152349.108 __zbx_zbx_setproctitle() title:'active checks #1 [getting list of active checks]'
18587:20181225:152349.108 In refresh_active_checks() host:'172.17.0.2' port:10051
18586:20181225:152349.087 agent #4 started[listener #3]
18586:20181225:152349.094 In zbx_tls_init_child()
18586:20181225:152349.100 OpenSSL library (version OpenSSL 1.0.1e-fips 11 Feb 2013) initialized
18586:20181225:152349.101 End of zbx_tls_init_child()
18586:20181225:152349.101 __zbx_zbx_setproctitle() title:'listener #3 [waiting for connection]'
18583:20181225:152349.104 agent #1 started [collector]
18583:20181225:152349.105 In init_cpu_collector()
18583:20181225:152349.105 End of init_cpu_collector():SUCCEED
18583:20181225:152349.105 __zbx_zbx_setproctitle() title:'collector [processing data]'
18583:20181225:152349.105 In update_cpustats()
18583:20181225:152349.105 End of update_cpustats()
18583:20181225:152349.105 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
18587:20181225:152349.105 agent #5 started [active checks #1]
18587:20181225:152349.105 In zbx_tls_init_child()
18587:20181225:152349.108 OpenSSL library (version OpenSSL 1.0.1e-fips 11 Feb 2013) initialized
18587:20181225:152349.108 End of zbx_tls_init_child()
18587:20181225:152349.108 In init_active_metrics()
18587:20181225:152349.108 buffer: first allocation for 100 elements
18587:20181225:152349.108 End of init_active_metrics()
18587:20181225:152349.108 In send_buffer() host:'172.17.0.2' port:10051 entries:0/100
18587:20181225:152349.108 End of send_buffer():SUCCEED
18587:20181225:152349.108 __zbx_zbx_setproctitle() title:'active checks #1 [getting list of active checks]'
18587:20181225:152349.108 In refresh_active_checks() host:'172.17.0.2' port:10051
18584:20181225:152349.109 agent #2 started[listener #1]
18584:20181225:152349.109 In zbx_tls_init_child()
18584:20181225:152349.112 OpenSSL library (version OpenSSL 1.0.1e-fips 11 Feb 2013) initialized
18584:20181225:152349.112 End of zbx_tls_init_child()
18584:20181225:152349.112 __zbx_zbx_setproctitle() title:'listener #1 [waiting for connection]'
18585:20181225:152349.112 OpenSSL library (version OpenSSL 1.0.1e-fips 11 Feb 2013) initialized
18585:20181225:152349.112 End of zbx_tls_init_child()
18585:20181225:152349.112 __zbx_zbx_setproctitle() title:'listener #2 [waiting for connection]'
18587:20181225:152349.112 sending [{"request":"active checks","host":"zabbix"}]
18587:20181225:152349.112 before read
18585:20181225:152349.460 __zbx_zbx_setproctitle() title:'listener #2 [processing request]'
18585:20181225:152349.461 Requested [system.cpu.util[,guest]]
18585:20181225:152349.461 Sending back [0.000000]
18585:20181225:152349.461 __zbx_zbx_setproctitle() title:'listener #2 [waiting for connection]'
18583:20181225:152350.106 __zbx_zbx_setproctitle() title:'collector [processing data]'
18583:20181225:152350.106 In update_cpustats()
18583:20181225:152350.106 End of update_cpustats()
18583:20181225:152350.106 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
18584:20181225:152350.463 __zbx_zbx_setproctitle() title:'listener #1 [processing request]'
18584:20181225:152350.463 Requested [system.cpu.util[,guest_nice]]
18584:20181225:152350.463 Sending back [0.000000]
18584:20181225:152350.463 __zbx_zbx_setproctitle() title:'listener #1 [waiting for connection]'
18583:20181225:152351.106 __zbx_zbx_setproctitle() title:'collector [processing data]'
18583:20181225:152351.106 In update_cpustats()
18583:20181225:152351.107 End of update_cpustats()
18583:20181225:152351.107 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
18586:20181225:152351.465 __zbx_zbx_setproctitle() title:'listener #3 [processing request]'
18586:20181225:152351.465 Requested [system.cpu.util[,idle]]
18586:20181225:152351.465 Sending back [90.355330]
18586:20181225:152351.466 __zbx_zbx_setproctitle() title:'listener #3 [waiting for connection]'
18583:20181225:152352.108 __zbx_zbx_setproctitle() title:'collector [processing data]'
18583:20181225:152352.108 In update_cpustats()
18583:20181225:152352.108 End of update_cpustats()
18583:20181225:152352.108 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
18585:20181225:152352.468 __zbx_zbx_setproctitle() title:'listener #2 [processing request]'
18585:20181225:152352.468 Requested [system.cpu.util[,interrupt]]
18585:20181225:152352.468 Sending back [0.000000]
18585:20181225:152352.469 __zbx_zbx_setproctitle() title:'listener #2 [waiting for connection]'
18583:20181225:152353.108 __zbx_zbx_setproctitle() title:'collector [processing data]'
18583:20181225:152353.108 In update_cpustats()
18583:20181225:152353.109 End of update_cpustats()
18583:20181225:152353.109 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
18584:20181225:152353.469 __zbx_zbx_setproctitle() title:'listener #1 [processing request]'
18584:20181225:152353.469 Requested [system.cpu.util[,iowait]]
18584:20181225:152353.469 Sending back [0.000000]
18584:20181225:152353.470 __zbx_zbx_setproctitle() title:'listener #1 [waiting for connection]'
18583:20181225:152354.109 __zbx_zbx_setproctitle() title:'collector [processing data]'
18583:20181225:152354.109 In update_cpustats()
18583:20181225:152354.109 End of update_cpustats()
18583:20181225:152354.109 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
18587:20181225:152354.109 active check configuration update from [172.17.0.2:10051] started to fail (ZBX_TCP_READ() timed out)
18587:20181225:152354.109 End of refresh_active_checks():FAIL
18587:20181225:152354.110 __zbx_zbx_setproctitle() title:'active checks #1 [processing active checks]'
18587:20181225:152354.110 In process_active_checks() server:'172.17.0.2' port:10051
18587:20181225:152354.110 End of process_active_checks()
18587:20181225:152354.110 In get_min_nextcheck()
18587:20181225:152354.110 End of get_min_nextcheck():-1
18587:20181225:152354.110 In send_buffer() host:'172.17.0.2' port:10051 entries:0/100
18587:20181225:152354.110 End of send_buffer():SUCCEED
18587:20181225:152354.110 __zbx_zbx_setproctitle() title:'active checks #1 [idle 1 sec]'
18586:20181225:152354.470 __zbx_zbx_setproctitle() title:'listener #3 [processing request]'
18586:20181225:152354.471 Requested [system.cpu.util[,nice]]
18586:20181225:152354.471 Sending back [0.000000]
18586:20181225:152354.471 __zbx_zbx_setproctitle() title:'listener #3 [waiting for connection]'
18583:20181225:152355.110 __zbx_zbx_setproctitle() title:'collector [processing data]'
18583:20181225:152355.110 In update_cpustats()
18583:20181225:152355.110 End of update_cpustats()
18583:20181225:152355.110 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'

anybody can help me?

[mehdiaghayari]

I tested, host network on docker with bellow command:

docker run --restart=always --name zabbix --network host -h zabbix -e ZBX_SERVER_NAME="zabbix" -p 80:80 -p 10051:10051 -d zabbix/zabbix-appliance:centos-trunk

so issue is resolved

please edit bellow manual or edit docker image for other people.

https://hub.docker.com/r/zabbix/zabbix-appliance/