Ad Widget

**Markku** · 20-10-2022, 20:31

First of all, thank you for providing detailed data!

Now, it looks like the server (170) acknowledges the 90 bytes of data sent by the agent, but doesn't do anything else. After three seconds (the default timeout set by the agent) the agent decides to close the connection. So it looks like a server-side issue, cannot tell much more.

What does the server log say?

If you like, you could see the actual data sent by the agent, if it gives you any new idea. There is a Wireshark dissector (https://github.com/markkuleinio/wire...bix-dissectors) if that is your cup of tea.

Markku

**cyber** · 21-10-2022, 10:33

2022/10/20 18:58:58.128487 [101] cannot connect to [192.168.24.169:10051]: dial tcp :0->192.168.24.169:10051: connect: connection refused
2022/10/20 18:58:58.128528 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:57799->192.168.24.170:10051: i/o timeout'

At least one of those should answer if you have a HA setup... If you say .170 is active currently, then there should not be a timeout...
I would add both server hosts to "Server" parameter also, not just locahost... At least allows you to run some testing against agent from server(s)

**Spectator** · 21-10-2022, 18:44

Thank you very much for your answers!

The Zabbix server is a native HA environment consisting of two nodes, as I mentioned earlier. The two nodes:

Code:

test-zbx-srv1 - 192.168.24.169
test-zbx-srv2 - 192.168.24.170

The linux host:

Code:

test-zbx-fe2 - 192.168.24.176

I stopped the zabbix-server service on test-zbx-srv1, so there will definitely only be traffic between test-zbx-srv2 (192.168.24.170) and the Linux host test-zbx-fe2 (192.168.24.176).

Unfortunately, the error was not resolved even when I entered the IP addresses of the two Zabbix servers in the Server=127.0.0.1 line in the linux host /etc/zabbix/zabbix-agent2.conf.
So that's how this line turned out:

Code:

Server=127.0.0.1,192.168.24.169,192.168.24.170

I removed Zabbix Agent2 from the linux host and installed the "standard" zabbix-agent (but no changes

).

The /etc/zabbix/zabbix-agentd.conf now looks like this:

Code:

PidFile=/run/zabbix/zabbix_agentd.pid
LogFile=/var/log/zabbix/zabbix_agentd.log
LogFileSize=0
Server=127.0.0.1,192.168.24.169,192.168.24.170
ServerActive=127.0.0.1;192.168.24.169;192.168.24.1 70 # I tried without 127.0.0.1, but no changes
Hostname=test-zbx-fe2
Include=/etc/zabbix/zabbix_agentd.d/*.conf

When starting zabbix-agentd on the linux host, it writes this to the log:

Code:

 1691:20221021:175132.798 Starting Zabbix Agent [test-zbx-fe2]. Zabbix 6.2.3 (revision 98ee88fc19d).
1691:20221021:175132.798 **** Enabled features ****
1691:20221021:175132.798 IPv6 support: YES
1691:20221021:175132.798 TLS support: YES
1691:20221021:175132.798 **************************
1691:20221021:175132.798 using configuration file: /etc/zabbix/zabbix_agentd.conf
1691:20221021:175132.801 agent #0 started [main process]
1694:20221021:175132.802 agent [NODE="1"]Home[/NODE] started [collector]
1695:20221021:175132.802 agent [NODE="2"]Forum[/NODE] started[listener #1]
1696:20221021:175132.802 agent #3 started[listener #2]
1697:20221021:175132.802 agent #4 started[listener #3]
1698:20221021:175132.805 agent #5 started [active checks #1]
1698:20221021:175132.808 Unable to connect to [192.168.24.169]:10051 [cannot connect to [[192.168.24.169]:10051]: [111] Connection refused]
1698:20221021:175135.808 Unable to receive from [192.168.24.170]:10051 [ZBX_TCP_READ() timed out]
1698:20221021:175135.808 Active check configuration update started to fail

When the agent starts on the linux host, the zabbix server writes nothing to the zabbix_server.log

If I telnet FROM THE LINUX HOST to port 10051 of the zabbix server:

Code:

[root@test-zbx-fe2 ~]# telnet 192.168.24.170 10051
Trying 192.168.24.170...
Connected to 192.168.24.170.
Escape character is '^]'.
quit
Connection closed by foreign host.
[root@test-zbx-fe2 ~]#

this appears in zabbix_server.log:

Code:

  3204:20221021:175928.458 Message from 192.168.24.176 is missing header. Message ignored.

If I telnet FROM THE LINUX HOST to port 10050 of the zabbix server:

Code:

[root@test-zbx-fe2 ~]# telnet 192.168.24.170 10050
Trying 192.168.24.170...
Connected to 192.168.24.170.
Escape character is '^]'.
Connection closed by foreign host.
[root@test-zbx-fe2 ~]#

Then it immediately returns the prompt like this, there is no trace in the zabbix server log - I think this is normal.

If I telnet FROM THE ZABBIX SERVER to port 10051 of the linux host:

Code:

[root@test-zbx-srv2 zabbix]# telnet 192.168.24.176 10051
Trying 192.168.24.176...
telnet: connect to address 192.168.24.176: Connection refused
[root@test-zbx-srv2 zabbix]#

This does not write anything to the linux host zabbix_agentd.log.

If I telnet FROM THE ZABBIX SERVER to port 10050 of the linux host:

Code:

[root@test-zbx-srv2 zabbix]# telnet 192.168.24.176 10050
Trying 192.168.24.176...
Connected to 192.168.24.176.
Escape character is '^]'.
quit
Connection closed by foreign host.
[root@test-zbx-srv2 zabbix]#

Then the linux host writes the following in zabbix_agentd.log:

Code:

  1695:20221021:180618.222 Message from 192.168.24.170 is missing header. Message ignored.

I don't think it can be a connection problem, what do you think?
Firewall is turned off on both machines, SELinux is disabled also.

Do you have any ideas?

**Markku** · 21-10-2022, 18:54

Sorry but none of that tries to explain why the server takes the connection from the active agent, but then decides to just hold the connection until agent times out.

Please increase the logging level in the server and show us the server log when the agent connects.

Markku

**Markku** · 21-10-2022, 19:01

Oops one more thing: did you tell your Zabbix server version? It must be 6.2.x with your 6.2.x agent.

Markku

**Spectator** · 21-10-2022, 19:13

Dear Markku,
I raised it the DebugLevel to 4 in the zabbix_server.conf and I restarted the Zabbix server.

After that, I also restarted zabbix-agentd on the linux host, and attached the zabbix_server.log generated in the meantime:.

Thank you

3353:20221021:190001.746 In zbx_ipc_async_socket_send()
3353:20221021:190001.746 In zbx_ipc_client_send() clientid:0
3353:20221021:190001.746 End of zbx_ipc_client_send():SUCCEED
3353:20221021:190001.746 End of zbx_ipc_async_socket_send():SUCCEED
3353:20221021:190001.746 In zbx_ipc_async_socket_flush() timeout:10
3353:20221021:190001.746 End of zbx_ipc_async_socket_flush():0
3352:20221021:190001.746 End of zbx_ipc_service_recv():1
3352:20221021:190001.746 In zbx_ha_dispatch_message()
3353:20221021:190001.746 In zbx_ipc_service_recv() timeout:0.997
3352:20221021:190001.746 In zbx_ipc_service_recv() timeout:1.000
3356:20221021:190001.975 End of zbx_ipc_service_recv():2
3356:20221021:190001.975 In zbx_ipc_service_recv() timeout:1.000
3379:20221021:190002.105 zbx_setproctitle() title:'self-monitoring [processing data]'
3379:20221021:190002.105 In collect_selfmon_stats()
3379:20221021:190002.105 End of collect_selfmon_stats()
3379:20221021:190002.105 zbx_setproctitle() title:'self-monitoring [processed data in 0.000096 sec, idle 1 sec]'
3382:20221021:190002.204 End of zbx_ipc_async_socket_recv():0
3382:20221021:190002.204 zbx_setproctitle() title:'poller Forum [got 0 values in 0.000080 sec, getting values]'
3382:20221021:190002.204 In get_values()
3382:20221021:190002.204 In DCconfig_get_poller_items() poller_type:0
3382:20221021:190002.204 End of DCconfig_get_poller_items():0
3382:20221021:190002.204 In DCconfig_get_poller_nextcheck() poller_type:0
3382:20221021:190002.204 End of DCconfig_get_poller_nextcheck():1666371632
3382:20221021:190002.204 End of get_values():0
3382:20221021:190002.204 zbx_setproctitle() title:'poller Forum [got 0 values in 0.000110 sec, idle 5 sec]'
3382:20221021:190002.204 In zbx_ipc_async_socket_recv() timeout:5
3384:20221021:190002.207 End of zbx_ipc_async_socket_recv():0
3384:20221021:190002.208 zbx_setproctitle() title:'poller #4 [got 0 values in 0.000084 sec, getting values]'
3384:20221021:190002.208 In get_values()
3384:20221021:190002.208 In DCconfig_get_poller_items() poller_type:0
3384:20221021:190002.208 End of DCconfig_get_poller_items():0
3384:20221021:190002.208 In DCconfig_get_poller_nextcheck() poller_type:0
3384:20221021:190002.208 End of DCconfig_get_poller_nextcheck():1666371632
3384:20221021:190002.208 End of get_values():0
3384:20221021:190002.208 zbx_setproctitle() title:'poller #4 [got 0 values in 0.000271 sec, idle 5 sec]'
3384:20221021:190002.208 In zbx_ipc_async_socket_recv() timeout:5
3366:20221021:190002.208 End of zbx_ipc_service_recv():2
3366:20221021:190002.208 In zbx_ipc_service_recv() timeout:1.000
3385:20221021:190002.208 End of zbx_ipc_async_socket_recv():0
3385:20221021:190002.208 zbx_setproctitle() title:'poller #5 [got 0 values in 0.000074 sec, getting values]'
3385:20221021:190002.208 In get_values()
3385:20221021:190002.208 In DCconfig_get_poller_items() poller_type:0
3385:20221021:190002.208 End of DCconfig_get_poller_items():0
3385:20221021:190002.208 In DCconfig_get_poller_nextcheck() poller_type:0
3385:20221021:190002.208 End of DCconfig_get_poller_nextcheck():1666371632
3385:20221021:190002.208 End of get_values():0
3385:20221021:190002.208 zbx_setproctitle() title:'poller #5 [got 0 values in 0.000167 sec, idle 5 sec]'
3385:20221021:190002.208 In zbx_ipc_async_socket_recv() timeout:5
3362:20221021:190002.210 End of zbx_ipc_service_recv():2
3362:20221021:190002.210 In zbx_ipc_service_recv() timeout:1.000
3383:20221021:190002.214 End of zbx_ipc_async_socket_recv():0
3383:20221021:190002.214 zbx_setproctitle() title:'poller #3 [got 0 values in 0.000077 sec, getting values]'
3383:20221021:190002.214 In get_values()
3383:20221021:190002.214 In DCconfig_get_poller_items() poller_type:0
3383:20221021:190002.214 End of DCconfig_get_poller_items():0
3383:20221021:190002.214 In DCconfig_get_poller_nextcheck() poller_type:0
3383:20221021:190002.214 End of DCconfig_get_poller_nextcheck():1666371632
3383:20221021:190002.214 End of get_values():0
3383:20221021:190002.214 zbx_setproctitle() title:'poller #3 [got 0 values in 0.000118 sec, idle 5 sec]'
3383:20221021:190002.214 In zbx_ipc_async_socket_recv() timeout:5
3381:20221021:190002.216 End of zbx_ipc_async_socket_recv():0
3381:20221021:190002.216 zbx_setproctitle() title:'poller Home [got 0 values in 0.000080 sec, getting values]'
3381:20221021:190002.216 In get_values()
3381:20221021:190002.216 In DCconfig_get_poller_items() poller_type:0
3381:20221021:190002.216 End of DCconfig_get_poller_items():0
3381:20221021:190002.216 In DCconfig_get_poller_nextcheck() poller_type:0
3381:20221021:190002.216 End of DCconfig_get_poller_nextcheck():1666371632
3381:20221021:190002.216 End of get_values():0
3381:20221021:190002.216 zbx_setproctitle() title:'poller Home [got 0 values in 0.000119 sec, idle 5 sec]'
3381:20221021:190002.216 In zbx_ipc_async_socket_recv() timeout:5
3373:20221021:190002.219 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000065 sec, syncing history]'
3373:20221021:190002.219 In zbx_sync_history_cache() history_num:0
3373:20221021:190002.219 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000073 sec, idle 1 sec]'
3374:20221021:190002.220 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000053 sec, syncing history]'
3374:20221021:190002.220 In zbx_sync_history_cache() history_num:0
3374:20221021:190002.220 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000085 sec, idle 1 sec]'
3376:20221021:190002.223 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000061 sec, syncing history]'
3376:20221021:190002.223 In zbx_sync_history_cache() history_num:0
3376:20221021:190002.223 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000080 sec, idle 1 sec]'
3375:20221021:190002.229 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000047 sec, syncing history]'
3375:20221021:190002.229 In zbx_sync_history_cache() history_num:0
3375:20221021:190002.229 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000060 sec, idle 1 sec]'
3399:20221021:190002.302 End of zbx_ipc_service_recv():2
3399:20221021:190002.302 In zbx_ipc_service_recv() timeout:1.000
3377:20221021:190002.352 zbx_setproctitle() title:'escalator Home [processed 0 escalations in 0.002574 sec, processing escalations]'
3377:20221021:190002.352 In process_escalations()
3377:20221021:190002.352 query [txnlev:0] [select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is not null and nextcheck<=1666371605 order by actionid,triggerid,itemid,escalationid]
3377:20221021:190002.353 End of process_escalations()
3377:20221021:190002.353 In process_escalations()
3377:20221021:190002.353 query [txnlev:0] [select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is not null and nextcheck<=1666371605 order by actionid,triggerid,itemid,escalationid]
3377:20221021:190002.353 End of process_escalations()
3377:20221021:190002.353 In process_escalations()
3377:20221021:190002.353 query [txnlev:0] [select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is null and serviceid is not null and nextcheck<=1666371605 order by actionid,triggerid,itemid,escalationid]
3377:20221021:190002.354 End of process_escalations()
3377:20221021:190002.354 In process_escalations()
3377:20221021:190002.354 query [txnlev:0] [select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is null and serviceid is null and nextcheck<=1666371605 order by actionid,triggerid,itemid,escalationid]
3377:20221021:190002.355 End of process_escalations()
3377:20221021:190002.355 zbx_setproctitle() title:'escalator Home [processed 0 escalations in 0.003041 sec, idle 3 sec]'
3393:20221021:190002.621 zbx_setproctitle() title:'alert syncer [queuing alerts]'
3393:20221021:190002.621 In am_db_get_alerts()
3393:20221021:190002.621 query [txnlev:1] [begin;]
3358:20221021:190002.621 End of zbx_ipc_service_recv():2
3358:20221021:190002.621 In zbx_ipc_service_recv() timeout:1.000
3393:20221021:190002.622 query [txnlev:1] [select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a. p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid]
3393:20221021:190002.623 query [txnlev:1] [commit;]
3393:20221021:190002.623 End of am_db_get_alerts():SUCCEED alerts:0
3393:20221021:190002.623 In am_db_flush_results()
3393:20221021:190002.623 In zbx_ipc_socket_write()
3393:20221021:190002.623 End of zbx_ipc_socket_write():SUCCEED
3393:20221021:190002.623 In zbx_ipc_socket_read()
3358:20221021:190002.623 End of zbx_ipc_service_recv():1
3358:20221021:190002.623 In am_flush_results() results:0
3358:20221021:190002.623 In zbx_ipc_client_send() clientid:6
3358:20221021:190002.623 End of zbx_ipc_client_send():SUCCEED
3358:20221021:190002.623 End of am_flush_results()
3358:20221021:190002.623 In zbx_ipc_service_recv() timeout:1.000
3393:20221021:190002.623 End of zbx_ipc_socket_read():SUCCEED
3393:20221021:190002.624 End of am_db_flush_results() flushed:0
3393:20221021:190002.624 zbx_setproctitle() title:'alert syncer [queued 0 alerts(s), flushed 0 result(s) in 0.002626 sec, idle 1 sec]'
3353:20221021:190002.745 End of zbx_ipc_service_recv():2
3353:20221021:190002.745 In ha_check_nodes() ha_status:active db_status:0
3353:20221021:190002.745 query [txnlev:1] [begin;]
3353:20221021:190002.746 query [txnlev:1] [select ha_nodeid,name,status,lastaccess,address,port,ha_s essionid from ha_node order by ha_nodeid for update]
3353:20221021:190002.746 query [txnlev:1] [select ha_failover_delay,auditlog_enabled from config]
3353:20221021:190002.747 In ha_db_get_time()
3353:20221021:190002.747 query [txnlev:1] [select cast(extract(epoch from now()) as int) from config]
3352:20221021:190002.747 End of zbx_ipc_service_recv():2
3352:20221021:190002.747 In zbx_ha_dispatch_message()
3352:20221021:190002.747 In zbx_ipc_service_recv() timeout:1.000
3353:20221021:190002.747 End of ha_db_get_time():SUCCEED db_time:1666371603
3353:20221021:190002.747 query [txnlev:1] [update ha_node set lastaccess=cast(extract(epoch from now()) as int) where ha_nodeid='cl9i6filr0001s08hnycqmytu']
3353:20221021:190002.748 query [txnlev:1] [commit;]
3353:20221021:190002.751 End of ha_check_nodes() nodeid:cl9i6filr0001s08hnycqmytu ha_status:active db_status:0
3353:20221021:190002.751 In zbx_ipc_async_socket_send()
3353:20221021:190002.751 In zbx_ipc_client_send() clientid:0
3353:20221021:190002.751 End of zbx_ipc_client_send():SUCCEED
3353:20221021:190002.751 End of zbx_ipc_async_socket_send():SUCCEED
3352:20221021:190002.751 End of zbx_ipc_service_recv():1
3352:20221021:190002.751 In zbx_ha_dispatch_message()
3352:20221021:190002.751 In zbx_ipc_service_recv() timeout:1.000
3353:20221021:190002.751 In zbx_ipc_async_socket_flush() timeout:10
3353:20221021:190002.751 End of zbx_ipc_async_socket_flush():0
3353:20221021:190002.751 In zbx_ipc_service_recv() timeout:0.998
3356:20221021:190002.976 End of zbx_ipc_service_recv():2
3356:20221021:190002.976 In zbx_ipc_service_recv() timeout:1.000
3379:20221021:190003.105 zbx_setproctitle() title:'self-monitoring [processing data]'
3379:20221021:190003.105 In collect_selfmon_stats()
3379:20221021:190003.105 End of collect_selfmon_stats()
3379:20221021:190003.105 zbx_setproctitle() title:'self-monitoring [processed data in 0.000095 sec, idle 1 sec]'
3388:20221021:190003.184 zbx_setproctitle() title:'trapper Forum [processing data]'
3388:20221021:190003.184 In zbx_ipc_async_socket_recv() timeout:0
3388:20221021:190003.184 End of zbx_ipc_async_socket_recv():0
3388:20221021:190003.184 trapper got '{"request":"active check heartbeat","host":"test-zbx-fe2","heartbeat_freq":60}'
3388:20221021:190003.184 In zbx_ipc_socket_open()
3388:20221021:190003.185 End of zbx_ipc_socket_open():SUCCEED
3388:20221021:190003.185 In zbx_ipc_socket_write()
3399:20221021:190003.185 In ipc_service_accept()
3388:20221021:190003.185 End of zbx_ipc_socket_write():SUCCEED
3399:20221021:190003.185 In ipc_service_add_client()
3388:20221021:190003.185 zbx_setproctitle() title:'trapper Forum [processed data in 0.000256 sec, waiting for connection]'
3399:20221021:190003.185 End of ipc_service_add_client() clientid:7
3399:20221021:190003.185 End of ipc_service_accept()
3399:20221021:190003.185 End of zbx_ipc_service_recv():2
3399:20221021:190003.185 In zbx_ipc_service_recv() timeout:1.000
3399:20221021:190003.185 End of zbx_ipc_service_recv():1
3399:20221021:190003.185 In zbx_ipc_service_recv() timeout:1.000
3388:20221021:190003.185 zbx_setproctitle() title:'trapper Forum [processing data]'
3388:20221021:190003.185 In zbx_ipc_async_socket_recv() timeout:0
3388:20221021:190003.185 End of zbx_ipc_async_socket_recv():0
3388:20221021:190003.185 trapper got '{"request":"active checks","host":"test-zbx-fe2"}'
3388:20221021:190003.185 In send_list_of_active_checks_json()
3388:20221021:190003.185 In is_ip4() ip:'192.168.24.175'
3388:20221021:190003.185 End of is_ip4():SUCCEED
3388:20221021:190003.185 In get_hostid_by_host() host:'test-zbx-fe2' metadata:''
3388:20221021:190003.185 query [txnlev:0] [select h.hostid,h.status,h.tls_accept,h.tls_issuer,h.tls_ subject,h.tls_psk_identity,a.host_metadata,a.liste n_ip,a.listen_dns,a.listen _port,a.flags from hosts h left join autoreg_host a on a.proxy_hostid is null and a.host=h.host where h.host='test-zbx-fe2' and h.status in (0,1) and h.flags<>2 and h.proxy_ho stid is null]
3366:20221021:190003.209 End of zbx_ipc_service_recv():2
3366:20221021:190003.209 In zbx_ipc_service_recv() timeout:1.000
3362:20221021:190003.211 End of zbx_ipc_service_recv():2
3362:20221021:190003.211 In zbx_ipc_service_recv() timeout:1.000
3373:20221021:190003.219 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000073 sec, syncing history]'
3373:20221021:190003.219 In zbx_sync_history_cache() history_num:0
3373:20221021:190003.219 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000072 sec, idle 1 sec]'
3374:20221021:190003.220 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000085 sec, syncing history]'
3374:20221021:190003.220 In zbx_sync_history_cache() history_num:0
3374:20221021:190003.220 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000095 sec, idle 1 sec]'
3376:20221021:190003.223 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000080 sec, syncing history]'
3376:20221021:190003.223 In zbx_sync_history_cache() history_num:0
3376:20221021:190003.223 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000068 sec, idle 1 sec]'
3375:20221021:190003.229 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000060 sec, syncing history]'
3375:20221021:190003.229 In zbx_sync_history_cache() history_num:0
3375:20221021:190003.229 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000059 sec, idle 1 sec]'
3393:20221021:190003.624 zbx_setproctitle() title:'alert syncer [queuing alerts]'
3393:20221021:190003.624 In am_db_get_alerts()
3393:20221021:190003.624 query [txnlev:1] [begin;]
3358:20221021:190003.625 End of zbx_ipc_service_recv():2
3358:20221021:190003.625 In zbx_ipc_service_recv() timeout:1.000
3393:20221021:190003.625 query [txnlev:1] [select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a. p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid]
3393:20221021:190003.626 query [txnlev:1] [commit;]
3393:20221021:190003.627 End of am_db_get_alerts():SUCCEED alerts:0
3393:20221021:190003.627 In am_db_flush_results()
3393:20221021:190003.627 In zbx_ipc_socket_write()
3393:20221021:190003.627 End of zbx_ipc_socket_write():SUCCEED
3393:20221021:190003.627 In zbx_ipc_socket_read()
3358:20221021:190003.627 End of zbx_ipc_service_recv():1
3358:20221021:190003.627 In am_flush_results() results:0
3358:20221021:190003.627 In zbx_ipc_client_send() clientid:6
3358:20221021:190003.627 End of zbx_ipc_client_send():SUCCEED
3358:20221021:190003.627 End of am_flush_results()
3393:20221021:190003.627 End of zbx_ipc_socket_read():SUCCEED
3393:20221021:190003.627 End of am_db_flush_results() flushed:0
3393:20221021:190003.627 zbx_setproctitle() title:'alert syncer [queued 0 alerts(s), flushed 0 result(s) in 0.003519 sec, idle 1 sec]'
3358:20221021:190003.627 In zbx_ipc_service_recv() timeout:1.000
3353:20221021:190003.752 End of zbx_ipc_service_recv():2
3353:20221021:190003.752 In zbx_ipc_async_socket_send()
3353:20221021:190003.752 In zbx_ipc_client_send() clientid:0
3353:20221021:190003.752 End of zbx_ipc_client_send():SUCCEED
3353:20221021:190003.752 End of zbx_ipc_async_socket_send():SUCCEED
3353:20221021:190003.752 In zbx_ipc_async_socket_flush() timeout:10
3353:20221021:190003.752 End of zbx_ipc_async_socket_flush():0
3353:20221021:190003.752 In zbx_ipc_service_recv() timeout:0.991
3352:20221021:190003.752 End of zbx_ipc_service_recv():1
3352:20221021:190003.752 In zbx_ha_dispatch_message()
3352:20221021:190003.752 In zbx_ipc_service_recv() timeout:1.000
3356:20221021:190003.977 End of zbx_ipc_service_recv():2
3356:20221021:190003.977 zbx_setproctitle() title:'service manager Home [processed 0 events, updated 0 event tags, deleted 0 problems, synced 0 service updates, idle 5.005526 sec during 5.010967 sec]'
3356:20221021:190003.977 In zbx_ipc_service_recv() timeout:1.000
3392:20221021:190004.095 zbx_setproctitle() title:'icmp pinger Home [getting values]'
3392:20221021:190004.095 In get_pinger_hosts()
3392:20221021:190004.095 In DCconfig_get_poller_items() poller_type:3
3392:20221021:190004.095 End of DCconfig_get_poller_items():0
3392:20221021:190004.095 End of get_pinger_hosts():0
3392:20221021:190004.095 In process_pinger_hosts()
3392:20221021:190004.095 End of process_pinger_hosts()
3392:20221021:190004.095 In DCconfig_get_poller_nextcheck() poller_type:3
3392:20221021:190004.095 End of DCconfig_get_poller_nextcheck():-1
3392:20221021:190004.095 zbx_setproctitle() title:'icmp pinger Home [got 0 values in 0.000231 sec, idle 5 sec]'
3379:20221021:190004.105 zbx_setproctitle() title:'self-monitoring [processing data]'
3379:20221021:190004.105 In collect_selfmon_stats()
3379:20221021:190004.105 End of collect_selfmon_stats()
3379:20221021:190004.105 zbx_setproctitle() title:'self-monitoring [processed data in 0.000104 sec, idle 1 sec]'
3399:20221021:190004.186 End of zbx_ipc_service_recv():2
3399:20221021:190004.186 In zbx_ipc_service_recv() timeout:1.000
3401:20221021:190004.205 End of zbx_ipc_async_socket_recv():0
3401:20221021:190004.206 zbx_setproctitle() title:'odbc poller Home [got 0 values in 0.000077 sec, getting values]'
3401:20221021:190004.206 In get_values()
3401:20221021:190004.206 In DCconfig_get_poller_items() poller_type:6
3401:20221021:190004.206 End of DCconfig_get_poller_items():0
3401:20221021:190004.206 In DCconfig_get_poller_nextcheck() poller_type:6
3401:20221021:190004.206 End of DCconfig_get_poller_nextcheck():-1
3401:20221021:190004.206 End of get_values():0
3401:20221021:190004.206 zbx_setproctitle() title:'odbc poller Home [got 0 values in 0.000120 sec, idle 5 sec]'
3401:20221021:190004.206 In zbx_ipc_async_socket_recv() timeout:5
3366:20221021:190004.210 End of zbx_ipc_service_recv():2
3366:20221021:190004.210 zbx_setproctitle() title:'lld manager Home [processed 0 LLD rules, idle 5.005546sec during 5.005574 sec]'
3366:20221021:190004.210 In zbx_ipc_service_recv() timeout:1.000
3362:20221021:190004.212 End of zbx_ipc_service_recv():2
3362:20221021:190004.212 In zbx_ipc_service_recv() timeout:1.000
3371:20221021:190004.214 zbx_setproctitle() title:'http poller Home [got 0 values in 0.001104 sec, getting values]'
3371:20221021:190004.215 In process_httptests()
3371:20221021:190004.215 query [txnlev:0] [select h.hostid,h.host,h.name,t.httptestid,t.name,t.agent ,t.authentication,t.http_user,t.http_password,t.ht tp_proxy,t.retries,t.ssl_c ert_file,t.ssl_key_file,t.ssl_key_password,t.verif y_peer,t.verify_host,t.delay from httptest t,hosts h where t.hostid=h.hostid and t.nextcheck<=1666371604 and mod(t.httptestid,1) =0 and t.status=0 and h.proxy_hostid is null and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0)]
3386:20221021:190004.215 End of zbx_ipc_async_socket_recv():0
3386:20221021:190004.215 zbx_setproctitle() title:'unreachable poller Home [got 0 values in 0.000101 sec, getting values]'
3386:20221021:190004.215 In get_values()
3386:20221021:190004.215 In DCconfig_get_poller_items() poller_type:1
3386:20221021:190004.215 End of DCconfig_get_poller_items():0
3386:20221021:190004.215 In DCconfig_get_poller_nextcheck() poller_type:1
3386:20221021:190004.215 End of DCconfig_get_poller_nextcheck():-1
3386:20221021:190004.215 End of get_values():0
3386:20221021:190004.215 zbx_setproctitle() title:'unreachable poller Home [got 0 values in 0.000122 sec, idle 5 sec]'
3386:20221021:190004.215 In zbx_ipc_async_socket_recv() timeout:5
3371:20221021:190004.216 End of process_httptests()
3371:20221021:190004.216 query [txnlev:0] [select min(t.nextcheck) from httptest t,hosts h where t.hostid=h.hostid and mod(t.httptestid,1)=0 and t.status=0 and h.proxy_hostid i s null and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0)]
3371:20221021:190004.217 No httptests to process in get_minnextcheck.
3371:20221021:190004.217 zbx_setproctitle() title:'http poller Home [got 0 values in 0.001597 sec, idle 5 sec]'
3373:20221021:190004.219 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000072 sec, syncing history]'
3373:20221021:190004.219 In zbx_sync_history_cache() history_num:0
3373:20221021:190004.219 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000088 sec, idle 1 sec]'
3374:20221021:190004.220 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000095 sec, syncing history]'
3374:20221021:190004.220 In zbx_sync_history_cache() history_num:0
3374:20221021:190004.220 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000086 sec, idle 1 sec]'
3376:20221021:190004.223 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000068 sec, syncing history]'
3376:20221021:190004.223 In zbx_sync_history_cache() history_num:0
3376:20221021:190004.223 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000065 sec, idle 1 sec]'
3375:20221021:190004.229 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000059 sec, syncing history]'
3375:20221021:190004.229 In zbx_sync_history_cache() history_num:0
3375:20221021:190004.229 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000064 sec, idle 1 sec]'
3378:20221021:190004.360 End of zbx_ipc_async_socket_recv():0
3378:20221021:190004.360 zbx_setproctitle() title:'proxy poller Home [exchanged data with 0 proxies in 0.000059 sec, exchanging data]'
3378:20221021:190004.360 In process_proxy()
3378:20221021:190004.360 In DCconfig_get_proxypoller_hosts()
3378:20221021:190004.360 End of DCconfig_get_proxypoller_hosts():0
3378:20221021:190004.360 End of process_proxy()
3378:20221021:190004.360 In DCconfig_get_proxypoller_nextcheck()
3378:20221021:190004.360 End of DCconfig_get_proxypoller_nextcheck():-1
3378:20221021:190004.360 zbx_setproctitle() title:'proxy poller Home [exchanged data with 0 proxies in 0.000085 sec, idle 5 sec]'
3378:20221021:190004.360 In zbx_ipc_async_socket_recv() timeout:5
3397:20221021:190004.375 End of zbx_ipc_async_socket_recv():0
3397:20221021:190004.375 zbx_setproctitle() title:'history poller #4 [got 0 values in 0.000076 sec, getting values]'
3397:20221021:190004.375 In get_values()
3397:20221021:190004.375 In DCconfig_get_poller_items() poller_type:5
3397:20221021:190004.375 End of DCconfig_get_poller_items():0
3397:20221021:190004.375 In DCconfig_get_poller_nextcheck() poller_type:5
3397:20221021:190004.375 End of DCconfig_get_poller_nextcheck():-1
3397:20221021:190004.375 End of get_values():0
3397:20221021:190004.375 zbx_setproctitle() title:'history poller #4 [got 0 values in 0.000100 sec, idle 5 sec]'
3397:20221021:190004.375 In zbx_ipc_async_socket_recv() timeout:5
3394:20221021:190004.376 End of zbx_ipc_async_socket_recv():0
3394:20221021:190004.376 zbx_setproctitle() title:'history poller Home [got 0 values in 0.000075 sec, getting values]'
3394:20221021:190004.376 In get_values()
3394:20221021:190004.376 In DCconfig_get_poller_items() poller_type:5
3394:20221021:190004.376 End of DCconfig_get_poller_items():0
3394:20221021:190004.376 In DCconfig_get_poller_nextcheck() poller_type:5
3394:20221021:190004.376 End of DCconfig_get_poller_nextcheck():-1
3394:20221021:190004.376 End of get_values():0
3394:20221021:190004.376 zbx_setproctitle() title:'history poller Home [got 0 values in 0.000127 sec, idle 5 sec]'
3394:20221021:190004.376 In zbx_ipc_async_socket_recv() timeout:5
3398:20221021:190004.380 End of zbx_ipc_async_socket_recv():0
3398:20221021:190004.380 zbx_setproctitle() title:'history poller #5 [got 0 values in 0.000077 sec, getting values]'
3398:20221021:190004.380 In get_values()
3398:20221021:190004.380 In DCconfig_get_poller_items() poller_type:5
3398:20221021:190004.380 End of DCconfig_get_poller_items():0
3398:20221021:190004.380 In DCconfig_get_poller_nextcheck() poller_type:5
3398:20221021:190004.381 End of DCconfig_get_poller_nextcheck():-1
3398:20221021:190004.381 End of get_values():0
3398:20221021:190004.381 zbx_setproctitle() title:'history poller #5 [got 0 values in 0.000144 sec, idle 5 sec]'
3398:20221021:190004.381 In zbx_ipc_async_socket_recv() timeout:5
3395:20221021:190004.397 End of zbx_ipc_async_socket_recv():0
3395:20221021:190004.397 zbx_setproctitle() title:'history poller Forum [got 0 values in 0.000075 sec, getting values]'
3395:20221021:190004.397 In get_values()
3395:20221021:190004.397 In DCconfig_get_poller_items() poller_type:5
3395:20221021:190004.397 End of DCconfig_get_poller_items():0
3395:20221021:190004.397 In DCconfig_get_poller_nextcheck() poller_type:5
3395:20221021:190004.397 End of DCconfig_get_poller_nextcheck():-1
3395:20221021:190004.397 End of get_values():0
3395:20221021:190004.397 zbx_setproctitle() title:'history poller Forum [got 0 values in 0.000103 sec, idle 5 sec]'
3395:20221021:190004.397 In zbx_ipc_async_socket_recv() timeout:5
3396:20221021:190004.403 End of zbx_ipc_async_socket_recv():0
3396:20221021:190004.403 zbx_setproctitle() title:'history poller #3 [got 0 values in 0.000075 sec, getting values]'
3396:20221021:190004.403 In get_values()
3396:20221021:190004.403 In DCconfig_get_poller_items() poller_type:5
3396:20221021:190004.403 End of DCconfig_get_poller_items():0
3396:20221021:190004.403 In DCconfig_get_poller_nextcheck() poller_type:5
3396:20221021:190004.403 End of DCconfig_get_poller_nextcheck():-1
3396:20221021:190004.403 End of get_values():0
3396:20221021:190004.403 zbx_setproctitle() title:'history poller #3 [got 0 values in 0.000116 sec, idle 5 sec]'
3396:20221021:190004.403 In zbx_ipc_async_socket_recv() timeout:5
3393:20221021:190004.627 zbx_setproctitle() title:'alert syncer [queuing alerts]'
3393:20221021:190004.627 In am_db_get_alerts()
3393:20221021:190004.627 query [txnlev:1] [begin;]
3358:20221021:190004.628 End of zbx_ipc_service_recv():2
3358:20221021:190004.628 zbx_setproctitle() title:'alert manager Home [sent 0, failed 0 alerts, idle 5.042607 sec during 5.043051 sec]'
3358:20221021:190004.628 In zbx_ipc_service_recv() timeout:1.000
3393:20221021:190004.628 query [txnlev:1] [select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a. p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid]
3393:20221021:190004.629 query [txnlev:1] [commit;]
3393:20221021:190004.629 End of am_db_get_alerts():SUCCEED alerts:0
3393:20221021:190004.629 In am_db_flush_results()
3393:20221021:190004.629 In zbx_ipc_socket_write()
3393:20221021:190004.629 End of zbx_ipc_socket_write():SUCCEED
3393:20221021:190004.629 In zbx_ipc_socket_read()
3358:20221021:190004.629 End of zbx_ipc_service_recv():1
3358:20221021:190004.629 In am_flush_results() results:0
3358:20221021:190004.630 In zbx_ipc_client_send() clientid:6
3358:20221021:190004.630 End of zbx_ipc_client_send():SUCCEED
3358:20221021:190004.630 End of am_flush_results()
3393:20221021:190004.630 End of zbx_ipc_socket_read():SUCCEED
3393:20221021:190004.630 End of am_db_flush_results() flushed:0
3358:20221021:190004.630 In zbx_ipc_service_recv() timeout:1.000
3393:20221021:190004.630 zbx_setproctitle() title:'alert syncer [queued 0 alerts(s), flushed 0 result(s) in 0.002438 sec, idle 1 sec]'
3353:20221021:190004.745 End of zbx_ipc_service_recv():2
3353:20221021:190004.745 In zbx_ipc_async_socket_send()
3353:20221021:190004.745 In zbx_ipc_client_send() clientid:0
3353:20221021:190004.745 End of zbx_ipc_client_send():SUCCEED
3353:20221021:190004.745 End of zbx_ipc_async_socket_send():SUCCEED
3353:20221021:190004.745 In zbx_ipc_async_socket_flush() timeout:10
3353:20221021:190004.745 End of zbx_ipc_async_socket_flush():0
3353:20221021:190004.745 In zbx_ipc_service_recv() timeout:0.998
3352:20221021:190004.745 End of zbx_ipc_service_recv():1
3352:20221021:190004.745 In zbx_ha_dispatch_message()
3352:20221021:190004.745 In zbx_ipc_service_recv() timeout:1.000
3356:20221021:190004.978 End of zbx_ipc_service_recv():2
3356:20221021:190004.979 In zbx_ipc_service_recv() timeout:1.000
3379:20221021:190005.105 zbx_setproctitle() title:'self-monitoring [processing data]'
3379:20221021:190005.105 In collect_selfmon_stats()
3379:20221021:190005.105 End of collect_selfmon_stats()
3379:20221021:190005.105 zbx_setproctitle() title:'self-monitoring [processed data in 0.000093 sec, idle 1 sec]'
3399:20221021:190005.187 End of zbx_ipc_service_recv():2
3399:20221021:190005.187 In zbx_ipc_service_recv() timeout:1.000
3366:20221021:190005.211 End of zbx_ipc_service_recv():2
3366:20221021:190005.211 In zbx_ipc_service_recv() timeout:1.000
3362:20221021:190005.213 End of zbx_ipc_service_recv():2
3362:20221021:190005.213 zbx_setproctitle() title:'preprocessing manager Home [queued 0, processed 0 values, idle 5.004843 sec during 5.004871 sec]'
3362:20221021:190005.213 In zbx_ipc_service_recv() timeout:1.000
3373:20221021:190005.219 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000088 sec, syncing history]'
3373:20221021:190005.219 In zbx_sync_history_cache() history_num:0
3373:20221021:190005.219 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000082 sec, idle 1 sec]'
3374:20221021:190005.220 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000086 sec, syncing history]'
3374:20221021:190005.220 In zbx_sync_history_cache() history_num:0
3374:20221021:190005.220 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000064 sec, idle 1 sec]'
3376:20221021:190005.223 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000065 sec, syncing history]'
3376:20221021:190005.223 In zbx_sync_history_cache() history_num:0
3376:20221021:190005.223 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000072 sec, idle 1 sec]'
3375:20221021:190005.229 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000064 sec, syncing history]'
3375:20221021:190005.229 In zbx_sync_history_cache() history_num:0
3375:20221021:190005.229 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000066 sec, idle 1 sec]'
3377:20221021:190005.355 zbx_setproctitle() title:'escalator Home [processed 0 escalations in 0.003041 sec, processing escalations]'
3377:20221021:190005.355 In process_escalations()
3377:20221021:190005.355 query [txnlev:0] [select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is not null and nextcheck<=1666371608 order by actionid,triggerid,itemid,escalationid]
3377:20221021:190005.356 End of process_escalations()
3377:20221021:190005.356 In process_escalations()
3377:20221021:190005.356 query [txnlev:0] [select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is not null and nextcheck<=1666371608 order by actionid,triggerid,itemid,escalationid]
3377:20221021:190005.357 End of process_escalations()
3377:20221021:190005.357 In process_escalations()
3377:20221021:190005.357 query [txnlev:0] [select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is null and serviceid is not null and nextcheck<=1666371608 order by actionid,triggerid,itemid,escalationid]
3377:20221021:190005.357 End of process_escalations()
3377:20221021:190005.357 In process_escalations()
3377:20221021:190005.357 query [txnlev:0] [select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is null and serviceid is null and nextcheck<=1666371608 order by actionid,triggerid,itemid,escalationid]
3377:20221021:190005.358 End of process_escalations()
3377:20221021:190005.358 zbx_setproctitle() title:'escalator Home [processed 0 escalations in 0.002839 sec, idle 3 sec]'
3380:20221021:190005.395 End of zbx_ipc_async_socket_recv():0
3380:20221021:190005.395 zbx_setproctitle() title:'task manager [processing tasks]'
3380:20221021:190005.395 query [txnlev:0] [select taskid,type,clock,ttl from task where status in (1,2) order by taskid]
3380:20221021:190005.396 zbx_setproctitle() title:'task manager [processed 0 task(s) in 0.001046 sec, idle 5 sec]'
3380:20221021:190005.396 In zbx_ipc_async_socket_recv() timeout:5
3393:20221021:190005.630 zbx_setproctitle() title:'alert syncer [queuing alerts]'
3393:20221021:190005.630 In am_db_get_alerts()
3393:20221021:190005.630 query [txnlev:1] [begin;]
3393:20221021:190005.631 query [txnlev:1] [select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a. p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid]
3358:20221021:190005.631 End of zbx_ipc_service_recv():2
3358:20221021:190005.631 In zbx_ipc_service_recv() timeout:1.000
3393:20221021:190005.632 query [txnlev:1] [commit;]
3393:20221021:190005.632 End of am_db_get_alerts():SUCCEED alerts:0
3393:20221021:190005.632 In am_db_flush_results()
3393:20221021:190005.632 In zbx_ipc_socket_write()
3393:20221021:190005.632 End of zbx_ipc_socket_write():SUCCEED
3393:20221021:190005.632 In zbx_ipc_socket_read()
3358:20221021:190005.632 End of zbx_ipc_service_recv():1
3358:20221021:190005.632 In am_flush_results() results:0
3358:20221021:190005.632 In zbx_ipc_client_send() clientid:6
3358:20221021:190005.632 End of zbx_ipc_client_send():SUCCEED
3358:20221021:190005.632 End of am_flush_results()
3358:20221021:190005.632 In zbx_ipc_service_recv() timeout:1.000
3393:20221021:190005.632 End of zbx_ipc_socket_read():SUCCEED
3393:20221021:190005.632 End of am_db_flush_results() flushed:0
3393:20221021:190005.632 zbx_setproctitle() title:'alert syncer [queued 0 alerts(s), flushed 0 result(s) in 0.002407 sec, idle 1 sec]'
3353:20221021:190005.745 End of zbx_ipc_service_recv():2
3353:20221021:190005.745 In zbx_ipc_async_socket_send()
3353:20221021:190005.745 In zbx_ipc_client_send() clientid:0
3353:20221021:190005.745 End of zbx_ipc_client_send():SUCCEED
3353:20221021:190005.745 End of zbx_ipc_async_socket_send():SUCCEED
3353:20221021:190005.745 In zbx_ipc_async_socket_flush() timeout:10
3353:20221021:190005.745 End of zbx_ipc_async_socket_flush():0
3353:20221021:190005.745 In zbx_ipc_service_recv() timeout:0.998
3352:20221021:190005.745 End of zbx_ipc_service_recv():1
3352:20221021:190005.745 In zbx_ha_dispatch_message()
3352:20221021:190005.745 In zbx_ipc_service_recv() timeout:1.000
3356:20221021:190005.980 End of zbx_ipc_service_recv():2
3356:20221021:190005.980 In zbx_ipc_service_recv() timeout:1.000
3379:20221021:190006.106 zbx_setproctitle() title:'self-monitoring [processing data]'
3379:20221021:190006.106 In collect_selfmon_stats()
3379:20221021:190006.106 End of collect_selfmon_stats()
3379:20221021:190006.106 zbx_setproctitle() title:'self-monitoring [processed data in 0.000094 sec, idle 1 sec]'
3399:20221021:190006.188 End of zbx_ipc_service_recv():2
3399:20221021:190006.188 In zbx_ipc_service_recv() timeout:1.000
3366:20221021:190006.212 End of zbx_ipc_service_recv():2
3366:20221021:190006.212 In zbx_ipc_service_recv() timeout:1.000
3362:20221021:190006.214 End of zbx_ipc_service_recv():2
3362:20221021:190006.214 In zbx_ipc_service_recv() timeout:1.000
3373:20221021:190006.219 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000082 sec, syncing history]'
3373:20221021:190006.219 In zbx_sync_history_cache() history_num:0
3373:20221021:190006.219 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000082 sec, idle 1 sec]'
3374:20221021:190006.221 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000064 sec, syncing history]'
3374:20221021:190006.221 In zbx_sync_history_cache() history_num:0
3374:20221021:190006.221 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000061 sec, idle 1 sec]'
3376:20221021:190006.223 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000072 sec, syncing history]'
3376:20221021:190006.223 In zbx_sync_history_cache() history_num:0
3376:20221021:190006.223 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000060 sec, idle 1 sec]'
3375:20221021:190006.229 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000066 sec, syncing history]'
3375:20221021:190006.229 In zbx_sync_history_cache() history_num:0
3375:20221021:190006.229 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000066 sec, idle 1 sec]'
3393:20221021:190006.632 zbx_setproctitle() title:'alert syncer [queuing alerts]'
3393:20221021:190006.632 In am_db_get_alerts()
3393:20221021:190006.632 query [txnlev:1] [begin;]
3358:20221021:190006.633 End of zbx_ipc_service_recv():2
3358:20221021:190006.633 In zbx_ipc_service_recv() timeout:1.000
3393:20221021:190006.633 query [txnlev:1] [select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a. p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid]
3393:20221021:190006.634 query [txnlev:1] [commit;]
3393:20221021:190006.635 End of am_db_get_alerts():SUCCEED alerts:0
3393:20221021:190006.635 In am_db_flush_results()
3393:20221021:190006.635 In zbx_ipc_socket_write()
3393:20221021:190006.635 End of zbx_ipc_socket_write():SUCCEED
3393:20221021:190006.635 In zbx_ipc_socket_read()
3358:20221021:190006.635 End of zbx_ipc_service_recv():1
3358:20221021:190006.635 In am_flush_results() results:0
3358:20221021:190006.635 In zbx_ipc_client_send() clientid:6
3358:20221021:190006.635 End of zbx_ipc_client_send():SUCCEED
3358:20221021:190006.635 End of am_flush_results()
3358:20221021:190006.635 In zbx_ipc_service_recv() timeout:1.000
3393:20221021:190006.635 End of zbx_ipc_socket_read():SUCCEED
3393:20221021:190006.635 End of am_db_flush_results() flushed:0
3393:20221021:190006.635 zbx_setproctitle() title:'alert syncer [queued 0 alerts(s), flushed 0 result(s) in 0.002748 sec, idle 1 sec]'
3353:20221021:190006.745 End of zbx_ipc_service_recv():2
3353:20221021:190006.745 In zbx_ipc_async_socket_send()
3353:20221021:190006.745 In zbx_ipc_client_send() clientid:0
3353:20221021:190006.746 End of zbx_ipc_client_send():SUCCEED
3353:20221021:190006.746 End of zbx_ipc_async_socket_send():SUCCEED
3353:20221021:190006.746 In zbx_ipc_async_socket_flush() timeout:10
3353:20221021:190006.746 End of zbx_ipc_async_socket_flush():0
3353:20221021:190006.746 In zbx_ipc_service_recv() timeout:0.998
3352:20221021:190006.746 End of zbx_ipc_service_recv():1
3352:20221021:190006.746 In zbx_ha_dispatch_message()
3352:20221021:190006.746 In zbx_ipc_service_recv() timeout:1.000
3356:20221021:190006.981 End of zbx_ipc_service_recv():2
3356:20221021:190006.981 In zbx_ipc_service_recv() timeout:1.000
3379:20221021:190007.106 zbx_setproctitle() title:'self-monitoring [processing data]'
3379:20221021:190007.106 In collect_selfmon_stats()
3379:20221021:190007.106 End of collect_selfmon_stats()
3379:20221021:190007.106 zbx_setproctitle() title:'self-monitoring [processed data in 0.000093 sec, idle 1 sec]'
3399:20221021:190007.189 End of zbx_ipc_service_recv():2
3399:20221021:190007.189 zbx_setproctitle() title:'availability manager Home [queued 0, processed 0 values, idle 5.888198 sec during 5.888227 sec]'
3399:20221021:190007.189 In zbx_ipc_service_recv() timeout:1.000
3382:20221021:190007.209 End of zbx_ipc_async_socket_recv():0
3382:20221021:190007.209 zbx_setproctitle() title:'poller Forum [got 0 values in 0.000110 sec, getting values]'
3382:20221021:190007.209 In get_values()
3382:20221021:190007.209 In DCconfig_get_poller_items() poller_type:0
3382:20221021:190007.209 End of DCconfig_get_poller_items():0
3382:20221021:190007.209 In DCconfig_get_poller_nextcheck() poller_type:0
3382:20221021:190007.209 End of DCconfig_get_poller_nextcheck():1666371632
3382:20221021:190007.209 End of get_values():0
3382:20221021:190007.209 zbx_setproctitle() title:'poller Forum [got 0 values in 0.000108 sec, idle 5 sec]'
3382:20221021:190007.209 In zbx_ipc_async_socket_recv() timeout:5
3384:20221021:190007.213 End of zbx_ipc_async_socket_recv():0
3384:20221021:190007.213 zbx_setproctitle() title:'poller #4 [got 0 values in 0.000271 sec, getting values]'
3384:20221021:190007.213 In get_values()
3384:20221021:190007.213 In DCconfig_get_poller_items() poller_type:0
3384:20221021:190007.213 End of DCconfig_get_poller_items():0
3384:20221021:190007.213 In DCconfig_get_poller_nextcheck() poller_type:0
3384:20221021:190007.213 End of DCconfig_get_poller_nextcheck():1666371632
3384:20221021:190007.213 End of get_values():0
3384:20221021:190007.213 zbx_setproctitle() title:'poller #4 [got 0 values in 0.000239 sec, idle 5 sec]'
3385:20221021:190007.213 End of zbx_ipc_async_socket_recv():0
3385:20221021:190007.213 zbx_setproctitle() title:'poller #5 [got 0 values in 0.000167 sec, getting values]'
3385:20221021:190007.213 In get_values()
3385:20221021:190007.213 In DCconfig_get_poller_items() poller_type:0
3385:20221021:190007.213 End of DCconfig_get_poller_items():0
3385:20221021:190007.213 In DCconfig_get_poller_nextcheck() poller_type:0
3385:20221021:190007.213 End of DCconfig_get_poller_nextcheck():1666371632
3385:20221021:190007.214 End of get_values():0
3385:20221021:190007.214 zbx_setproctitle() title:'poller #5 [got 0 values in 0.000282 sec, idle 5 sec]'
3385:20221021:190007.214 In zbx_ipc_async_socket_recv() timeout:5
3366:20221021:190007.214 End of zbx_ipc_service_recv():2
3366:20221021:190007.214 In zbx_ipc_service_recv() timeout:1.000
3384:20221021:190007.214 In zbx_ipc_async_socket_recv() timeout:5
3362:20221021:190007.215 End of zbx_ipc_service_recv():2
3362:20221021:190007.215 In zbx_ipc_service_recv() timeout:1.000
3383:20221021:190007.217 End of zbx_ipc_async_socket_recv():0
3383:20221021:190007.217 zbx_setproctitle() title:'poller #3 [got 0 values in 0.000118 sec, getting values]'
3383:20221021:190007.217 In get_values()
3383:20221021:190007.217 In DCconfig_get_poller_items() poller_type:0
3383:20221021:190007.217 End of DCconfig_get_poller_items():0
3383:20221021:190007.217 In DCconfig_get_poller_nextcheck() poller_type:0
3383:20221021:190007.217 End of DCconfig_get_poller_nextcheck():1666371632
3383:20221021:190007.217 End of get_values():0
3383:20221021:190007.217 zbx_setproctitle() title:'poller #3 [got 0 values in 0.000157 sec, idle 5 sec]'
3383:20221021:190007.217 In zbx_ipc_async_socket_recv() timeout:5
3373:20221021:190007.219 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000082 sec, syncing history]'
3373:20221021:190007.220 In zbx_sync_history_cache() history_num:0
3373:20221021:190007.220 zbx_setproctitle() title:'history syncer Home [processed 0 values, 0 triggers in 0.000088 sec, idle 1 sec]'
3374:20221021:190007.221 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000061 sec, syncing history]'
3374:20221021:190007.221 In zbx_sync_history_cache() history_num:0
3374:20221021:190007.221 zbx_setproctitle() title:'history syncer Forum [processed 0 values, 0 triggers in 0.000067 sec, idle 1 sec]'
3381:20221021:190007.221 End of zbx_ipc_async_socket_recv():0
3381:20221021:190007.221 zbx_setproctitle() title:'poller Home [got 0 values in 0.000119 sec, getting values]'
3381:20221021:190007.221 In get_values()
3381:20221021:190007.221 In DCconfig_get_poller_items() poller_type:0
3381:20221021:190007.221 End of DCconfig_get_poller_items():0
3381:20221021:190007.221 In DCconfig_get_poller_nextcheck() poller_type:0
3381:20221021:190007.221 End of DCconfig_get_poller_nextcheck():1666371632
3381:20221021:190007.221 End of get_values():0
3381:20221021:190007.221 zbx_setproctitle() title:'poller Home [got 0 values in 0.000120 sec, idle 5 sec]'
3381:20221021:190007.221 In zbx_ipc_async_socket_recv() timeout:5
3376:20221021:190007.224 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000060 sec, syncing history]'
3376:20221021:190007.224 In zbx_sync_history_cache() history_num:0
3376:20221021:190007.224 zbx_setproctitle() title:'history syncer #4 [processed 0 values, 0 triggers in 0.000084 sec, idle 1 sec]'
3375:20221021:190007.229 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000066 sec, syncing history]'
3375:20221021:190007.229 In zbx_sync_history_cache() history_num:0
3375:20221021:190007.230 zbx_setproctitle() title:'history syncer #3 [processed 0 values, 0 triggers in 0.000078 sec, idle 1 sec]'
3393:20221021:190007.635 zbx_setproctitle() title:'alert syncer [queuing alerts]'
3393:20221021:190007.635 In am_db_get_alerts()
3393:20221021:190007.635 query [txnlev:1] [begin;]
3358:20221021:190007.636 End of zbx_ipc_service_recv():2
3393:20221021:190007.636 query [txnlev:1] [select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a. p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid]
3358:20221021:190007.636 In zbx_ipc_service_recv() timeout:1.000
3393:20221021:190007.637 query [txnlev:1] [commit;]
3393:20221021:190007.637 End of am_db_get_alerts():SUCCEED alerts:0
3393:20221021:190007.638 In am_db_flush_results()
3393:20221021:190007.638 In zbx_ipc_socket_write()
3393:20221021:190007.638 End of zbx_ipc_socket_write():SUCCEED
3393:20221021:190007.638 In zbx_ipc_socket_read()
3358:20221021:190007.638 End of zbx_ipc_service_recv():1
3358:20221021:190007.638 In am_flush_results() results:0
3358:20221021:190007.638 In zbx_ipc_client_send() clientid:6
3358:20221021:190007.638 End of zbx_ipc_client_send():SUCCEED
3358:20221021:190007.638 End of am_flush_results()
3393:20221021:190007.638 End of zbx_ipc_socket_read():SUCCEED
3393:20221021:190007.638 End of am_db_flush_results() flushed:0
3393:20221021:190007.638 zbx_setproctitle() title:'alert syncer [queued 0 alerts(s), flushed 0 result(s) in 0.002551 sec, idle 1 sec]'
3358:20221021:190007.638 In zbx_ipc_service_recv() timeout:1.000
3353:20221021:190007.746 End of zbx_ipc_service_recv():2
3353:20221021:190007.746 In ha_check_nodes() ha_status:active db_status:0
3353:20221021:190007.746 query [txnlev:1] [begin;]
3352:20221021:190007.747 End of zbx_ipc_service_recv():2
3352:20221021:190007.747 In zbx_ha_dispatch_message()
3352:20221021:190007.747 In zbx_ipc_service_recv() timeout:1.000
3353:20221021:190007.748 query [txnlev:1] [select ha_nodeid,name,status,lastaccess,address,port,ha_s essionid from ha_node order by ha_nodeid for update]
3353:20221021:190007.749 query [txnlev:1] [select ha_failover_delay,auditlog_enabled from config]
3353:20221021:190007.749 In ha_db_get_time()
3353:20221021:190007.749 query [txnlev:1] [select cast(extract(epoch from now()) as int) from config]
3353:20221021:190007.750 End of ha_db_get_time():SUCCEED db_time:1666371608
3353:20221021:190007.750 query [txnlev:1] [update ha_node set lastaccess=cast(extract(epoch from now()) as int) where ha_nodeid='cl9i6filr0001s08hnycqmytu']
3353:20221021:190007.750 query [txnlev:1] [commit;]
3353:20221021:190007.751 End of ha_check_nodes() nodeid:cl9i6filr0001s08hnycqmytu ha_status:active db_status:0
3353:20221021:190007.751 In zbx_ipc_async_socket_send()
3353:20221021:190007.751 In zbx_ipc_client_send() clientid:0
3353:20221021:190007.752 End of zbx_ipc_client_send():SUCCEED
3353:20221021:190007.752 End of zbx_ipc_async_socket_send():SUCCEED
3353:20221021:190007.752 In zbx_ipc_async_socket_flush() timeout:10
3353:20221021:190007.752 End of zbx_ipc_async_socket_flush():0
3353:20221021:190007.752 In zbx_ipc_service_recv() timeout:0.997
3352:20221021:190007.752 End of zbx_ipc_service_recv():1
3352:20221021:190007.752 In zbx_ha_dispatch_message()
3352:20221021:190007.752 In zbx_ipc_service_recv() timeout:1.000
3356:20221021:190007.982 End of zbx_ipc_service_recv():2
3356:20221021:190007.982 In zbx_ipc_service_recv() timeout:1.000
3379:20221021:190008.106 zbx_setproctitle() title:'self-monitoring [processing data]'
3379:20221021:190008.106 In collect_selfmon_stats()
3379:20221021:190008.106 End of collect_selfmon_stats()
3379:20221021:190008.106 zbx_setproctitle() title:'self-monitoring [processed data in 0.000093 sec, idle 1 sec]'

**Spectator** · 21-10-2022, 19:26

Originally posted by Markku

Oops one more thing: did you tell your Zabbix server version? It must be 6.2.x with your 6.2.x agent.

Markku

The Zabbix server version:

Code:

[root@test-zbx-srv2 zabbix]# zabbix_server --version
zabbix_server (Zabbix) 6.2.3
Revision 98ee88fc19d 21 September 2022, compilation time: Sep 21 2022 00:00:00

Copyright (C) 2022 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 3.0.1 14 Dec 2021
Running with OpenSSL 3.0.1 14 Dec 2021
[root@test-zbx-srv2 zabbix]#

The linux host Zabbix Agent version:

Code:

[root@test-zbx-fe2 ~]# zabbix_agentd --version
zabbix_agentd (daemon) (Zabbix) 6.2.3
Revision 98ee88fc19d 21 September 2022, compilation time: Sep 21 2022 00:00:00

Copyright (C) 2022 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 3.0.1 14 Dec 2021
Running with OpenSSL 3.0.1 14 Dec 2021
[root@test-zbx-fe2 ~]#

Everything is 6.2.3 version.

**Markku** · 21-10-2022, 19:30

Small question: who is 192.168.24.175?

Markku

**Spectator** · 21-10-2022, 19:44

Originally posted by Markku

Small question: who is 192.168.24.175?

Markku

Your question is valid

I accidentally changed agent2 to agent1 on another linux host. This host is test-zbx-fe1, IP address: 192.168.24.175
I deleted the test-zbx-fe2 host from the Zabbix web interface and added test-zbx-fe1 in the same way. It was the same mistake about him.
But in order not to confuse you here, when I wrote my posts here, I changed everything back to the name and IP address of the test-zbx-fe2 linux host - except for the one you found

So instead of .175, imagine .176

**Markku** · 21-10-2022, 20:35

This got my attention (process 3388 = trapper logs):

3388:20221021:190003.184 zbx_setproctitle() title:'trapper Forum [processing data]'
3388:20221021:190003.184 In zbx_ipc_async_socket_recv() timeout:0
3388:20221021:190003.184 End of zbx_ipc_async_socket_recv():0
3388:20221021:190003.184 trapper got '{"request":"active check heartbeat","host":"test-zbx-fe2","heartbeat_freq":60}'
3388:20221021:190003.184 In zbx_ipc_socket_open()
3388:20221021:190003.185 End of zbx_ipc_socket_open():SUCCEED
3388:20221021:190003.185 In zbx_ipc_socket_write()
3388:20221021:190003.185 End of zbx_ipc_socket_write():SUCCEED
3388:20221021:190003.185 zbx_setproctitle() title:'trapper Forum [processed data in 0.000256 sec, waiting for connection]'
3388:20221021:190003.185 zbx_setproctitle() title:'trapper Forum [processing data]'
3388:20221021:190003.185 In zbx_ipc_async_socket_recv() timeout:0
3388:20221021:190003.185 End of zbx_ipc_async_socket_recv():0
3388:20221021:190003.185 trapper got '{"request":"active checks","host":"test-zbx-fe2"}'
3388:20221021:190003.185 In send_list_of_active_checks_json()
3388:20221021:190003.185 In is_ip4() ip:'192.168.24.175'
3388:20221021:190003.185 End of is_ip4():SUCCEED
3388:20221021:190003.185 In get_hostid_by_host() host:'test-zbx-fe2' metadata:''
3388:20221021:190003.185 query [txnlev:0] [select h.hostid,h.status,h.tls_accept,h.tls_issuer,h.tls_ subject,h.tls_psk_identity,a.host_metadata,a.liste n_ip,a.listen_dns,a.listen _port,a.flags from hosts h left join autoreg_host a on a.proxy_hostid is null and a.host=h.host where h.host='test-zbx-fe2' and h.status in (0,1) and h.flags<>2 and h.proxy_ho stid is null]

and that's it, like it never got any response from the database for that query (after receiving the JSON requests from the agent). Does your database work normally without suspicious messages in database log or zabbix_server.log?

What was the error message on the agent log at the same time?

Markku

**Spectator** · 21-10-2022, 23:57

Dear Markku,

This is very strange, I just don't know what to make of this problem

If I create an active proxy on which I install Zabbix-Agent2, the proxy works fine, it can measure itself with the "Linux by Zabbix agent active" template.
But neither the proxy nor the Zabbix Server can communicate directly with the linux host to be measured, on which I am running an active agent.

You gave me some ideas.
A three-node PostgreSQL HA cluster is the database server behind Zabbix, HAProxy maintains a virtual cluster IP address, Zabbix communicate during this virtual IP to the database.
The PostgreSLQ cluster was configured in such a way that it can only be connected to it via TLS. I turned this off, but the problem did not improve.
TimeScaleDB was also installed, version 2.8.1, it is installed by default with the system, unfortunately I cannot downgrade it, even though the Zabbix Server log indicated that this version is not supported:

Code:

  4072:20221021:225829.232 using configuration file: /etc/zabbix/zabbix_server.conf
  4072:20221021:225829.334 Unsupported DB! timescaledb version is 20801 which is higher than maximum of 20799
  4072:20221021:225829.335 Recommended version should not be higher than TimescaleDB Community Edition 2.7.
  4072:20221021:225829.335 Detected license [apache] does not support compression. Compression is supported in TimescaleDB Community Edition.
  4072:20221021:225829.359 current database version (mandatory/optional): 06020000/06020002
  4072:20221021:225829.359 required mandatory version: 06020000

With TimeScaleDB higher than version 2.7, Zabbix Server started only if I enabled this in zabbix_server.conf:

Code:

AllowUnsupportedDBVersions=1

I deleted the entire Zabbix database and recreated it without TimeScaleDB, so now I only have PostgreSQL (version 14.5).
Unfortunately, the situation has not improved.
I don't understand what kind of database problem it could be, since Zabbix is working, the measurements of the two Zabbix Servers are working, if I add a new host, it can save it in the database...

The zabbix_agent2.log file of the linux host contains these from the agent startup:

Code:

2022/10/21 23:20:32.469163 Starting Zabbix Agent 2 (6.2.3)
2022/10/21 23:20:32.470487 OpenSSL library (OpenSSL 3.0.1 14 Dec 2021) initialized
2022/10/21 23:20:32.470554 using configuration file: /etc/zabbix/zabbix_agent2.conf
2022/10/21 23:20:32.470648 using plugin 'Agent' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470670 using plugin 'Ceph' (built-in) providing following interfaces: exporter, runner, configurator
2022/10/21 23:20:32.470691 using plugin 'Cpu' (built-in) providing following interfaces: exporter, collector, runner
2022/10/21 23:20:32.470700 using plugin 'DNS' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470714 using plugin 'Docker' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470722 using plugin 'File' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470731 using plugin 'Hw' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470738 using plugin 'Kernel' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470744 using plugin 'Log' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470750 using plugin 'MQTT' (built-in) providing following interfaces: watcher, configurator
2022/10/21 23:20:32.470756 using plugin 'Memcached' (built-in) providing following interfaces: exporter, runner, configurator
2022/10/21 23:20:32.470763 using plugin 'Memory' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470769 using plugin 'Modbus' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470775 using plugin 'Mysql' (built-in) providing following interfaces: exporter, runner, configurator
2022/10/21 23:20:32.470785 using plugin 'NetIf' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470797 using plugin 'Oracle' (built-in) providing following interfaces: exporter, runner, configurator
2022/10/21 23:20:32.470807 using plugin 'Postgres' (built-in) providing following interfaces: exporter, runner, configurator
2022/10/21 23:20:32.470832 using plugin 'Proc' (built-in) providing following interfaces: exporter, collector
2022/10/21 23:20:32.470840 using plugin 'ProcExporter' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470855 using plugin 'Redis' (built-in) providing following interfaces: exporter, runner, configurator
2022/10/21 23:20:32.470863 using plugin 'Smart' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470869 using plugin 'Sw' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470876 using plugin 'Swap' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470891 using plugin 'SystemRun' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470899 using plugin 'Systemd' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470906 using plugin 'TCP' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470912 using plugin 'UDP' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470919 using plugin 'Uname' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470925 using plugin 'Uptime' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470931 using plugin 'Users' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470938 using plugin 'VFSDev' (built-in) providing following interfaces: exporter, collector
2022/10/21 23:20:32.470944 using plugin 'VFSDir' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470951 using plugin 'VfsFs' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470958 using plugin 'WebCertificate' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470964 using plugin 'WebPage' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470974 using plugin 'ZabbixAsync' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.470984 using plugin 'ZabbixStats' (built-in) providing following interfaces: exporter, configurator
2022/10/21 23:20:32.470989 lowering the plugin ZabbixSync capacity to 1 as the configured capacity 100 exceeds limits
2022/10/21 23:20:32.470995 using plugin 'ZabbixSync' (built-in) providing following interfaces: exporter
2022/10/21 23:20:32.471122 Plugin support version 1.0
2022/10/21 23:20:32.471149 Zabbix Agent2 hostname: [test-zbx-fe2]
2022/10/21 23:20:36.472556 [101] cannot connect to [192.168.24.169:10051]: dial tcp :0->192.168.24.169:10051: connect: connection refused
2022/10/21 23:20:36.472590 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:36177->192.168.24.170:10051: i/o timeout'
2022/10/21 23:20:36.472596 [101] active check configuration update from host [test-zbx-fe2] started to fail
2022/10/21 23:22:40.473209 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:34779->192.168.24.170:10051: i/o timeout'
2022/10/21 23:22:40.473302 [101] active check configuration update from host [test-zbx-fe2] started to fail
2022/10/21 23:24:44.474533 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:60943->192.168.24.170:10051: i/o timeout'
2022/10/21 23:24:44.474539 [101] active check configuration update from host [test-zbx-fe2] started to fail
2022/10/21 23:26:48.472478 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:48283->192.168.24.170:10051: i/o timeout'
2022/10/21 23:26:48.472531 [101] active check configuration update from host [test-zbx-fe2] started to fail
2022/10/21 23:28:52.472655 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:55297->192.168.24.170:10051: i/o timeout'
2022/10/21 23:28:52.472691 [101] active check configuration update from host [test-zbx-fe2] started to fail
2022/10/21 23:30:56.472626 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:51265->192.168.24.170:10051: i/o timeout'
2022/10/21 23:30:56.472698 [101] active check configuration update from host [test-zbx-fe2] started to fail
2022/10/21 23:33:00.472815 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:41561->192.168.24.170:10051: i/o timeout'
2022/10/21 23:33:00.472851 [101] active check configuration update from host [test-zbx-fe2] started to fail
2022/10/21 23:35:04.472807 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:46983->192.168.24.170:10051: i/o timeout'
2022/10/21 23:35:04.472849 [101] active check configuration update from host [test-zbx-fe2] started to fail
2022/10/21 23:37:08.472669 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:41371->192.168.24.170:10051: i/o timeout'
2022/10/21 23:37:08.472725 [101] active check configuration update from host [test-zbx-fe2] started to fail
2022/10/21 23:39:12.472665 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:39691->192.168.24.170:10051: i/o timeout'
2022/10/21 23:39:12.472736 [101] active check configuration update from host [test-zbx-fe2] started to fail
2022/10/21 23:41:16.472530 [101] cannot receive data from [192.168.24.170:10051]: Cannot read message: 'read tcp 192.168.24.176:57479->192.168.24.170:10051: i/o timeout'
2022/10/21 23:41:16.472591 [101] active check configuration update from host [test-zbx-fe2] started to fail

In the meantime, in the zabbix_server.log file of the Zabbix Server is what I copied earlier (you quoted from it in your previous post).

The max_connections value of the database is set to 100, I think this should be enough.
I'm totally at a loss...

**Spectator** · 22-10-2022, 00:19

The database server log from when I restart zabbix-agent2 on the linux host and it prints this first error:

Code:

active check configuration update from host [test-zbx-fe2] started to fail

[13991] [63531813.36a7] [2022-10-22 00:07:15.231 CEST] [0]: LOG: statement: select oid from pg_type where typname='bytea'
[13991] [63531813.36a7] [2022-10-22 00:07:15.232 CEST] [0]: LOG: statement: set escape_string_warning to off
[13991] [63531813.36a7] [2022-10-22 00:07:15.232 CEST] [0]: LOG: statement: set extra_float_digits to 3
[13991] [63531813.36a7] [2022-10-22 00:07:15.232 CEST] [0]: LOG: statement: show standard_conforming_strings
[13991] [63531813.36a7] [2022-10-22 00:07:15.233 CEST] [0]: LOG: statement: set bytea_output=escape
[997] [635306b1.3e5] [2022-10-22 00:07:15.575 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[12383] [6353162b.305f] [2022-10-22 00:07:15.711 CEST] [0]: LOG: statement: select taskid,type,clock,ttl from task where status in (1,2) order by taskid
[12381] [6353162b.305d] [2022-10-22 00:07:16.233 CEST] [0]: LOG: statement: begin;
[12381] [6353162b.305d] [2022-10-22 00:07:16.234 CEST] [0]: LOG: statement: select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a.p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid
[12381] [6353162b.305d] [2022-10-22 00:07:16.235 CEST] [0]: LOG: statement: commit;
[12395] [6353162b.306b] [2022-10-22 00:07:16.614 CEST] [0]: LOG: statement: select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is not null and nextcheck<=1666390039 order by actionid,triggerid,itemid,escalationid
[12395] [6353162b.306b] [2022-10-22 00:07:16.615 CEST] [0]: LOG: statement: select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is not null and nextcheck<=1666390039 order by actionid,triggerid,itemid,escalationid
[12395] [6353162b.306b] [2022-10-22 00:07:16.616 CEST] [0]: LOG: statement: select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is null and serviceid is not null and nextcheck<=1666390039 order by actionid,triggerid,itemid,escalationid
[12395] [6353162b.306b] [2022-10-22 00:07:16.616 CEST] [0]: LOG: statement: select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is null and serviceid is null and nextcheck<=1666390039 order by actionid,triggerid,itemid,escalationid
[12377] [6353162b.3059] [2022-10-22 00:07:16.621 CEST] [0]: LOG: statement: begin;
[12377] [6353162b.3059] [2022-10-22 00:07:16.621 CEST] [0]: LOG: statement: insert into history (itemid,clock,ns,value) values (42256,1666390036,594629586,335.71246573520648);

[12377] [6353162b.3059] [2022-10-22 00:07:16.621 CEST] [2129132]: LOG: statement: commit;
[12377] [6353162b.3059] [2022-10-22 00:07:16.622 CEST] [0]: LOG: statement: begin;
[12377] [6353162b.3059] [2022-10-22 00:07:16.623 CEST] [0]: LOG: statement: commit;
[12377] [6353162b.3059] [2022-10-22 00:07:16.623 CEST] [0]: LOG: statement: begin;
[12377] [6353162b.3059] [2022-10-22 00:07:16.623 CEST] [0]: LOG: statement: commit;
[12358] [63531626.3046] [2022-10-22 00:07:16.786 CEST] [0]: LOG: statement: begin;
[12358] [63531626.3046] [2022-10-22 00:07:16.787 CEST] [0]: LOG: statement: select ha_nodeid,name,status,lastaccess,address,port,ha_s essionid from ha_node order by ha_nodeid for update
[12358] [63531626.3046] [2022-10-22 00:07:16.787 CEST] [2129133]: LOG: statement: select ha_failover_delay,auditlog_enabled from config
[12358] [63531626.3046] [2022-10-22 00:07:16.789 CEST] [2129133]: LOG: statement: select cast(extract(epoch from now()) as int) from config
[12358] [63531626.3046] [2022-10-22 00:07:16.790 CEST] [2129133]: LOG: statement: update ha_node set lastaccess=cast(extract(epoch from now()) as int) where ha_nodeid='cl9izno1500012w8g1p3rjj52'
[12358] [63531626.3046] [2022-10-22 00:07:16.790 CEST] [2129133]: LOG: statement: commit;
[997] [635306b1.3e5] [2022-10-22 00:07:16.796 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[12320] [6353161c.3020] [2022-10-22 00:07:16.799 CEST] [0]: LOG: statement: begin;
[12320] [6353161c.3020] [2022-10-22 00:07:16.799 CEST] [0]: LOG: statement: select ha_nodeid,name,status,lastaccess,address,port,ha_s essionid from ha_node order by ha_nodeid for update
[12320] [6353161c.3020] [2022-10-22 00:07:16.800 CEST] [2129134]: LOG: statement: select ha_failover_delay,auditlog_enabled from config
[12320] [6353161c.3020] [2022-10-22 00:07:16.800 CEST] [2129134]: LOG: statement: select cast(extract(epoch from now()) as int) from config
[12320] [6353161c.3020] [2022-10-22 00:07:16.801 CEST] [2129134]: LOG: statement: update ha_node set lastaccess=cast(extract(epoch from now()) as int) where ha_nodeid='cl9izke0b00019f8h9da74ubj'
[12320] [6353161c.3020] [2022-10-22 00:07:16.801 CEST] [2129134]: LOG: statement: commit;
[12381] [6353162b.305d] [2022-10-22 00:07:17.235 CEST] [0]: LOG: statement: begin;
[12381] [6353162b.305d] [2022-10-22 00:07:17.236 CEST] [0]: LOG: statement: select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a.p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid
[12381] [6353162b.305d] [2022-10-22 00:07:17.237 CEST] [0]: LOG: statement: commit;
[997] [635306b1.3e5] [2022-10-22 00:07:17.272 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[12398] [6353162b.306e] [2022-10-22 00:07:17.420 CEST] [0]: LOG: statement: select h.hostid,h.host,h.name,t.httptestid,t.name,t.agent ,t.authentication,t.http_user,t.http_password,t.ht tp_proxy,t.retries,t.ssl_cert_file,t.ssl_key_file, t.ssl_key_password,t.verify_peer,t.verify_host,t.d elay from httptest t,hosts h where t.hostid=h.hostid and t.nextcheck<=1666390037 and mod(t.httptestid,1)=0 and t.status=0 and h.proxy_hostid is null and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0)
[12398] [6353162b.306e] [2022-10-22 00:07:17.421 CEST] [0]: LOG: statement: select min(t.nextcheck) from httptest t,hosts h where t.hostid=h.hostid and mod(t.httptestid,1)=0 and t.status=0 and h.proxy_hostid is null and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0)
[997] [635306b1.3e5] [2022-10-22 00:07:17.503 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[997] [635306b1.3e5] [2022-10-22 00:07:17.518 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[997] [635306b1.3e5] [2022-10-22 00:07:17.518 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[12378] [6353162b.305a] [2022-10-22 00:07:17.621 CEST] [0]: LOG: statement: begin;
[12378] [6353162b.305a] [2022-10-22 00:07:17.622 CEST] [0]: LOG: statement: insert into history (itemid,clock,ns,value) values (42257,1666390037,595662748,0);

[12378] [6353162b.305a] [2022-10-22 00:07:17.622 CEST] [2129135]: LOG: statement: commit;
[12378] [6353162b.305a] [2022-10-22 00:07:17.623 CEST] [0]: LOG: statement: begin;
[12378] [6353162b.305a] [2022-10-22 00:07:17.623 CEST] [0]: LOG: statement: commit;
[12378] [6353162b.305a] [2022-10-22 00:07:17.624 CEST] [0]: LOG: statement: begin;
[12378] [6353162b.305a] [2022-10-22 00:07:17.624 CEST] [0]: LOG: statement: commit;
[12390] [6353162b.3066] [2022-10-22 00:07:17.927 CEST] [0]: LOG: statement: select h.hostid,h.status,h.tls_accept,h.tls_issuer,h.tls_ subject,h.tls_psk_identity,a.host_metadata,a.liste n_ip,a.listen_dns,a.listen_port,a.flags from hosts h left join autoreg_host a on a.proxy_hostid is null and a.host=h.host where h.host='test-zbx-fe2' and h.status in (0,1) and h.flags<>2 and h.proxy_hostid is null
[12381] [6353162b.305d] [2022-10-22 00:07:18.237 CEST] [0]: LOG: statement: begin;
[12381] [6353162b.305d] [2022-10-22 00:07:18.238 CEST] [0]: LOG: statement: select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a.p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid
[12381] [6353162b.305d] [2022-10-22 00:07:18.239 CEST] [0]: LOG: statement: commit;
[997] [635306b1.3e5] [2022-10-22 00:07:18.578 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[12376] [6353162b.3058] [2022-10-22 00:07:18.622 CEST] [0]: LOG: statement: begin;
[12376] [6353162b.3058] [2022-10-22 00:07:18.622 CEST] [0]: LOG: statement: insert into history (itemid,clock,ns,value) values (42258,1666390038,596694919,0);

[12376] [6353162b.3058] [2022-10-22 00:07:18.623 CEST] [2129136]: LOG: statement: commit;
[12376] [6353162b.3058] [2022-10-22 00:07:18.624 CEST] [0]: LOG: statement: begin;
[12376] [6353162b.3058] [2022-10-22 00:07:18.624 CEST] [0]: LOG: statement: commit;
[12376] [6353162b.3058] [2022-10-22 00:07:18.625 CEST] [0]: LOG: statement: begin;
[12376] [6353162b.3058] [2022-10-22 00:07:18.625 CEST] [0]: LOG: statement: commit;
[997] [635306b1.3e5] [2022-10-22 00:07:18.759 CEST] [0]: LOG: statement: SELECT CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), 0, CASE WHEN latest_end_lsn IS NULL THEN NULL ELSE received_tli END, slot_name, conninfo, NULL FROM pg_catalog.pg_stat_get_wal_receiver()
[14003] [63531817.36b3] [2022-10-22 00:07:19.089 CEST] [0]: LOG: statement: SET search_path='public'
[14003] [63531817.36b3] [2022-10-22 00:07:19.090 CEST] [0]: LOG: statement: SET bytea_output=escape
[14003] [63531817.36b3] [2022-10-22 00:07:19.090 CEST] [0]: LOG: statement: SHOW server_version
[14003] [63531817.36b3] [2022-10-22 00:07:19.090 CEST] [0]: LOG: statement: select oid,typname from pg_type
[14003] [63531817.36b3] [2022-10-22 00:07:19.092 CEST] [0]: LOG: statement: SELECT datname, usename, ssl, client_addr, cipher FROM pg_stat_ssl JOIN pg_stat_activity ON pg_stat_ssl.pid=pg_stat_activity.pid AND pg_stat_activity.usename='zabbix'
[14003] [63531817.36b3] [2022-10-22 00:07:19.095 CEST] [0]: LOG: statement: SELECT 1 FROM information_schema.tables WHERE table_catalog='tzabbixdb' AND table_schema='public' AND table_name='dbversion'
[14003] [63531817.36b3] [2022-10-22 00:07:19.097 CEST] [0]: LOG: statement: SELECT dv.mandatory FROM dbversion dv
[14003] [63531817.36b3] [2022-10-22 00:07:19.098 CEST] [0]: LOG: statement: SELECT NULL FROM config c
[14003] [63531817.36b3] [2022-10-22 00:07:19.099 CEST] [0]: LOG: statement: SELECT han.ha_nodeid,han.address,han.port,han.status FROM ha_node han ORDER BY han.lastaccess DESC
[14003] [63531817.36b3] [2022-10-22 00:07:19.100 CEST] [0]: LOG: statement: SELECT c.configid,c.default_theme,c.show_technical_errors ,c.severity_color_0,c.severity_color_1,c.severity_ color_2,c.severity_color_3,c.severity_color_4,c.se verity_color_5,c.custom_color,c.problem_unack_colo r,c.problem_ack_color,c.ok_unack_color,c.ok_ack_co lor,c.default_lang,c.x_frame_options,c.default_tim ezone,c.session_key,c.dbversion_status FROM config c
[14003] [63531817.36b3] [2022-10-22 00:07:19.101 CEST] [0]: LOG: statement: BEGIN
[14003] [63531817.36b3] [2022-10-22 00:07:19.101 CEST] [0]: LOG: statement: SELECT sessionid,userid,lastaccess FROM sessions WHERE sessionid='4b29b85e3a76b15c53d83483a7addd93' AND status=0
[14003] [63531817.36b3] [2022-10-22 00:07:19.102 CEST] [0]: LOG: statement: SELECT userid,username,name,surname,url,autologin,autolog out,lang,refresh,theme,attempt_failed,attempt_ip,a ttempt_clock,rows_per_page,timezone,roleid FROM users WHERE userid=1
[14003] [63531817.36b3] [2022-10-22 00:07:19.103 CEST] [0]: LOG: statement: SELECT g.debug_mode,g.users_status,g.gui_access,g.userdir ectoryid FROM usrgrp g,users_groups ug WHERE g.usrgrpid=ug.usrgrpid AND ug.userid=1
[14003] [63531817.36b3] [2022-10-22 00:07:19.104 CEST] [0]: LOG: statement: SELECT type FROM role WHERE roleid='3'
[14003] [63531817.36b3] [2022-10-22 00:07:19.104 CEST] [0]: LOG: statement: COMMIT
[14003] [63531817.36b3] [2022-10-22 00:07:19.105 CEST] [0]: LOG: statement: BEGIN
[14003] [63531817.36b3] [2022-10-22 00:07:19.106 CEST] [0]: LOG: statement: SELECT r.roleid,r.name,r.type FROM role r WHERE r.roleid='3'
[14003] [63531817.36b3] [2022-10-22 00:07:19.106 CEST] [0]: LOG: statement: SELECT role_ruleid,roleid,type,name,value_int,value_str,v alue_moduleid,value_serviceid FROM role_rule WHERE roleid=3
[14003] [63531817.36b3] [2022-10-22 00:07:19.107 CEST] [0]: LOG: statement: SELECT md.moduleid FROM module md WHERE md.status=1
[14003] [63531817.36b3] [2022-10-22 00:07:19.108 CEST] [0]: LOG: statement: COMMIT
[14003] [63531817.36b3] [2022-10-22 00:07:19.109 CEST] [0]: LOG: statement: SELECT md.moduleid,md.id,md.relative_path,md.config FROM module md WHERE md.status=1 ORDER BY md.relative_path
[14003] [63531817.36b3] [2022-10-22 00:07:19.109 CEST] [0]: LOG: statement: SELECT NULL FROM users u WHERE u.userid=1 FOR UPDATE
[14003] [63531817.36b3] [2022-10-22 00:07:19.110 CEST] [0]: LOG: statement: SELECT type,value_id,value_int,value_str,idx2 FROM profiles WHERE userid=1 AND idx='web.sidebar.mode'
[14003] [63531817.36b3] [2022-10-22 00:07:19.111 CEST] [0]: LOG: statement: BEGIN
[14003] [63531817.36b3] [2022-10-22 00:07:19.111 CEST] [0]: LOG: statement: SELECT c.configid,c.default_theme,c.search_limit,c.max_in _table,c.server_check_interval,c.work_period,c.sho w_technical_errors,c.history_period,c.period_defau lt,c.max_period,c.severity_color_0,c.severity_colo r_1,c.severity_color_2,c.severity_color_3,c.severi ty_color_4,c.severity_color_5,c.severity_name_0,c. severity_name_1,c.severity_name_2,c.severity_name_ 3,c.severity_name_4,c.severity_name_5,c.custom_col or,c.ok_period,c.blink_period,c.problem_unack_colo r,c.problem_ack_color,c.ok_unack_color,c.ok_ack_co lor,c.problem_unack_style,c.problem_ack_style,c.ok _unack_style,c.ok_ack_style,c.discovery_groupid,c. default_inventory_mode,c.alert_usrgrpid,c.snmptrap _logging,c.default_lang,c.default_timezone,c.login _attempts,c.login_block,c.validate_uri_schemes,c.u ri_valid_schemes,c.x_frame_options,c.iframe_sandbo xing_enabled,c.iframe_sandboxing_exceptions,c.max_ overview_table_size,c.connect_timeout,c.socket_tim eout,c.media_type_test_timeout,c.script_timeout,c. item_test_timeout,c.url,c.report_test_timeout,c.au ditlog_enabled,c.ha_failover_delay,c.geomaps_tile_ provider,c.geomaps_tile_url,c.geomaps_max_zoom,c.g eomaps_attribution,c.vault_provider FROM config c
[14003] [63531817.36b3] [2022-10-22 00:07:19.112 CEST] [0]: LOG: statement: COMMIT
[14003] [63531817.36b3] [2022-10-22 00:07:19.113 CEST] [0]: LOG: statement: SELECT han.ha_nodeid,han.address,han.port,han.lastaccess FROM ha_node han WHERE han.status=3 ORDER BY han.lastaccess DESC LIMIT 1
[14003] [63531817.36b3] [2022-10-22 00:07:19.113 CEST] [0]: LOG: statement: BEGIN
[14003] [63531817.36b3] [2022-10-22 00:07:19.113 CEST] [0]: LOG: statement: ROLLBACK
[12381] [6353162b.305d] [2022-10-22 00:07:19.239 CEST] [0]: LOG: statement: begin;
[12381] [6353162b.305d] [2022-10-22 00:07:19.240 CEST] [0]: LOG: statement: select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a.p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid
[12381] [6353162b.305d] [2022-10-22 00:07:19.241 CEST] [0]: LOG: statement: commit;
[12395] [6353162b.306b] [2022-10-22 00:07:19.617 CEST] [0]: LOG: statement: select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is not null and nextcheck<=1666390042 order by actionid,triggerid,itemid,escalationid
[12395] [6353162b.306b] [2022-10-22 00:07:19.618 CEST] [0]: LOG: statement: select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is not null and nextcheck<=1666390042 order by actionid,triggerid,itemid,escalationid
[12395] [6353162b.306b] [2022-10-22 00:07:19.618 CEST] [0]: LOG: statement: select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is null and serviceid is not null and nextcheck<=1666390042 order by actionid,triggerid,itemid,escalationid
[12395] [6353162b.306b] [2022-10-22 00:07:19.619 CEST] [0]: LOG: statement: select escalationid,actionid,triggerid,eventid,r_eventid, nextcheck,esc_step,status,itemid,acknowledgeid,ser vicealarmid,serviceid from escalations where triggerid is null and itemid is null and serviceid is null and nextcheck<=1666390042 order by actionid,triggerid,itemid,escalationid
[12379] [6353162b.305b] [2022-10-22 00:07:19.624 CEST] [0]: LOG: statement: begin;
[12379] [6353162b.305b] [2022-10-22 00:07:19.624 CEST] [0]: LOG: statement: insert into history (itemid,clock,ns,value) values (42259,1666390039,597765600,0);

[12379] [6353162b.305b] [2022-10-22 00:07:19.625 CEST] [2129138]: LOG: statement: commit;
[12379] [6353162b.305b] [2022-10-22 00:07:19.625 CEST] [0]: LOG: statement: begin;
[12379] [6353162b.305b] [2022-10-22 00:07:19.626 CEST] [0]: LOG: statement: commit;
[12379] [6353162b.305b] [2022-10-22 00:07:19.626 CEST] [0]: LOG: statement: begin;
[12379] [6353162b.305b] [2022-10-22 00:07:19.627 CEST] [0]: LOG: statement: commit;
[997] [635306b1.3e5] [2022-10-22 00:07:19.801 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[12381] [6353162b.305d] [2022-10-22 00:07:20.241 CEST] [0]: LOG: statement: begin;
[12381] [6353162b.305d] [2022-10-22 00:07:20.242 CEST] [0]: LOG: statement: select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a.p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid
[12381] [6353162b.305d] [2022-10-22 00:07:20.243 CEST] [0]: LOG: statement: commit;
[997] [635306b1.3e5] [2022-10-22 00:07:20.275 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[997] [635306b1.3e5] [2022-10-22 00:07:20.506 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[997] [635306b1.3e5] [2022-10-22 00:07:20.521 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[997] [635306b1.3e5] [2022-10-22 00:07:20.522 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[12383] [6353162b.305f] [2022-10-22 00:07:20.717 CEST] [0]: LOG: statement: select taskid,type,clock,ttl from task where status in (1,2) order by taskid
[12381] [6353162b.305d] [2022-10-22 00:07:21.243 CEST] [0]: LOG: statement: begin;
[12381] [6353162b.305d] [2022-10-22 00:07:21.244 CEST] [0]: LOG: statement: select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a.p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid
[12381] [6353162b.305d] [2022-10-22 00:07:21.245 CEST] [0]: LOG: statement: commit;
[997] [635306b1.3e5] [2022-10-22 00:07:21.581 CEST] [0]: LOG: statement: SELECT pg_catalog.pg_postmaster_start_time(), CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE ('x' || pg_catalog.substr(pg_catalog.pg_walfile_name(pg_ca talog.pg_current_wal_lsn()), 1, 8))::bit(32)::int END, CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 ELSE pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_w al_lsn(), '0/0')::bigint END, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_last_wal_ replay_lsn(), '0/0')::bigint, pg_catalog.pg_wal_lsn_diff(COALESCE(pg_catalog.pg_ last_wal_receive_lsn(), '0/0'), '0/0')::bigint, pg_catalog.pg_is_in_recovery() AND pg_catalog.pg_is_wal_replay_paused(), pg_catalog.pg_last_xact_replay_timestamp(), pg_catalog.array_to_json(pg_catalog.array_agg(pg_c atalog.row_to_json(ri))) FROM (SELECT (SELECT rolname FROM pg_authid WHERE oid = usesysid) AS usename, application_name, client_addr, w.state, sync_state, sync_priority FROM pg_catalog.pg_stat_get_wal_senders() w, pg_catalog.pg_stat_get_activity(pid)) AS ri
[12377] [6353162b.3059] [2022-10-22 00:07:21.624 CEST] [0]: LOG: statement: begin;
[12377] [6353162b.3059] [2022-10-22 00:07:21.625 CEST] [0]: LOG: statement: insert into history (itemid,clock,ns,value) values (42261,1666390041,599978197,371.27328223368801);

[12377] [6353162b.3059] [2022-10-22 00:07:21.625 CEST] [2129139]: LOG: statement: commit;
[12377] [6353162b.3059] [2022-10-22 00:07:21.626 CEST] [0]: LOG: statement: begin;
[12377] [6353162b.3059] [2022-10-22 00:07:21.627 CEST] [0]: LOG: statement: commit;
[12377] [6353162b.3059] [2022-10-22 00:07:21.627 CEST] [0]: LOG: statement: begin;
[12377] [6353162b.3059] [2022-10-22 00:07:21.627 CEST] [0]: LOG: statement: commit;
[12358] [63531626.3046] [2022-10-22 00:07:21.786 CEST] [0]: LOG: statement: begin;
[12358] [63531626.3046] [2022-10-22 00:07:21.787 CEST] [0]: LOG: statement: select ha_nodeid,name,status,lastaccess,address,port,ha_s essionid from ha_node order by ha_nodeid for update
[12358] [63531626.3046] [2022-10-22 00:07:21.787 CEST] [2129140]: LOG: statement: select ha_failover_delay,auditlog_enabled from config
[12358] [63531626.3046] [2022-10-22 00:07:21.788 CEST] [2129140]: LOG: statement: select cast(extract(epoch from now()) as int) from config
[12358] [63531626.3046] [2022-10-22 00:07:21.788 CEST] [2129140]: LOG: statement: update ha_node set lastaccess=cast(extract(epoch from now()) as int) where ha_nodeid='cl9izno1500012w8g1p3rjj52'
[12358] [63531626.3046] [2022-10-22 00:07:21.789 CEST] [2129140]: LOG: statement: commit;
[12320] [6353161c.3020] [2022-10-22 00:07:21.799 CEST] [0]: LOG: statement: begin;
[12320] [6353161c.3020] [2022-10-22 00:07:21.800 CEST] [0]: LOG: statement: select ha_nodeid,name,status,lastaccess,address,port,ha_s essionid from ha_node order by ha_nodeid for update
[12320] [6353161c.3020] [2022-10-22 00:07:21.800 CEST] [2129141]: LOG: statement: select ha_failover_delay,auditlog_enabled from config
[12320] [6353161c.3020] [2022-10-22 00:07:21.801 CEST] [2129141]: LOG: statement: select cast(extract(epoch from now()) as int) from config
[12320] [6353161c.3020] [2022-10-22 00:07:21.801 CEST] [2129141]: LOG: statement: update ha_node set lastaccess=cast(extract(epoch from now()) as int) where ha_nodeid='cl9izke0b00019f8h9da74ubj'
[12320] [6353161c.3020] [2022-10-22 00:07:21.801 CEST] [2129141]: LOG: statement: commit;
[12381] [6353162b.305d] [2022-10-22 00:07:22.246 CEST] [0]: LOG: statement: begin;
[12381] [6353162b.305d] [2022-10-22 00:07:22.246 CEST] [0]: LOG: statement: select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a.p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid
[12381] [6353162b.305d] [2022-10-22 00:07:22.247 CEST] [0]: LOG: statement: commit;

**Markku** · 22-10-2022, 08:39

I hope the long logs don't scare away some of the readers

(maybe attach them as text files next time?)

Let's look at your situation again.

You said the only item working is the "Active agent availability". That's an item that basically tracks if the active agent has been successfully sending the heatbeats to the server. So that detail is working. Other than that, the agent logs show that the agent is not able to get the active checks list from the server, thus it is unable to start monitoring anything.

That is apparently (without knowing the contents of the packets here!) shown also in the packet capture you showed: the first TCP session was about the active checks (it got waiting and timed out) and the second TCP session (that worked fine) was the heartbeat (no server response required).

So the question remains: why doesn't the server respond to the active checks request?

You also said:

If I create an active proxy on which I install Zabbix-Agent2, the proxy works fine, it can measure itself with the "Linux by Zabbix agent active" template. But neither the proxy nor the Zabbix Server can communicate directly with the linux host to be measured, on which I am running an active agent.

That means that the server is very much capable of handling the trapper messages coming in, from the proxy in this case.

Did you try monitoring the agent of the proxy directly from the server (not via the proxy), does that work?

Is that test-zbx-fe2 your only active agent otherwise?

Markku

**Spectator** · 22-10-2022, 23:25

Dear Markku,

Thanks for your response!
You are right about reading the long log, but unfortunately it was not allowed to be uploaded here as an attachment due to its size.

I did some more tests:
1) I increased the PostgreSQL database max_connections and HAProxy max_connections values from 100 to 200 - unfortunately no change.
2) Instead of the VIP address and port of the PostgreSQL cluster, I changed the settings of the two Zabbix servers and the web frontend machine directly to the IP address of the active node and the TCP port 5432 - unfortunately, there is no change.
3) I added the linux host with a plain "Linux by Zabbix agent" template, when the Zabbix server queries the linux host agent and not the linux host agent sends the data to the zabbix server - in this case everything works perfectly. I just want the active agent to send the data to the zabbix server, right?
4) If I want to monitor the linux host with an active agent via the active proxy, then "Active agent availability" is unknown(0) under the linux host latest data.
In this case, the linux host setting:
host name: test-zbx-fe2
template: Linux by Zabbix agent active
Interface: Agent, 192.168.24.176 (this is the linux host IP)
Monitored by proxy: test-zbx-proxy1
On the linux host (test-zbx-fe2), the proxy IP address and the IP addresses of the two Zabbix servers are added to the Server= and ServerActive= lines in the agent configuration file.

The contents of the zabbix_proxy.log on the proxy (Nothing appears related to the test-zbx-fe2 linux host):

Code:

170567:20221022:230546.276 Starting Zabbix Proxy (active) [test-zbx-proxy1]. Zabbix 6.2.3 (revision 98ee88fc19d).
170567:20221022:230546.277 **** Enabled features ****
170567:20221022:230546.277 SNMP monitoring:       YES
170567:20221022:230546.277 IPMI monitoring:       YES
170567:20221022:230546.277 Web monitoring:        YES
170567:20221022:230546.277 VMware monitoring:     YES
170567:20221022:230546.277 ODBC:                  YES
170567:20221022:230546.277 SSH support:           YES
170567:20221022:230546.277 IPv6 support:          YES
170567:20221022:230546.277 TLS support:           YES
170567:20221022:230546.277 **************************
170567:20221022:230546.277 using configuration file: /etc/zabbix/zabbix_proxy.conf
170567:20221022:230546.285 current database version (mandatory/optional): 06020000/06020002
170567:20221022:230546.285 required mandatory version: 06020000
170567:20221022:230546.289 proxy #0 started [main process]
170570:20221022:230546.289 proxy #1 started [configuration syncer #1]
170570:20221022:230546.299 Unable to connect to [192.168.24.169]:10051 [cannot connect to [[192.168.24.169]:10051]: [111] Connection refused]
170572:20221022:230546.299 proxy #2 started [trapper #1]
170573:20221022:230546.300 proxy #3 started [trapper #2]
170574:20221022:230546.300 proxy #4 started [trapper #3]
170577:20221022:230546.302 proxy #7 started [preprocessing manager #1]
170582:20221022:230546.303 proxy #12 started [data sender #1]
170579:20221022:230546.303 proxy #9 started [preprocessing worker #2]
170584:20221022:230546.303 proxy #14 started [http poller #1]
170585:20221022:230546.304 proxy #15 started [http poller #2]
170575:20221022:230546.304 proxy #5 started [trapper #4]
170578:20221022:230546.304 proxy #8 started [preprocessing worker #1]
170586:20221022:230546.305 proxy #16 started [discoverer #1]
170583:20221022:230546.305 proxy #13 started [housekeeper #1]
170576:20221022:230546.306 proxy #6 started [trapper #5]
170580:20221022:230546.307 proxy #10 started [preprocessing worker #3]
170587:20221022:230546.308 proxy #17 started [discoverer #2]
170588:20221022:230546.309 proxy #18 started [history syncer #1]
170589:20221022:230546.310 proxy #19 started [history syncer #2]
170592:20221022:230546.310 proxy #22 started [self-monitoring #1]
170593:20221022:230546.310 proxy #23 started [task manager #1]
170594:20221022:230546.311 proxy #24 started [poller #1]
170581:20221022:230546.311 proxy #11 started [heartbeat sender #1]
170595:20221022:230546.312 proxy #25 started [poller #2]
170590:20221022:230546.314 proxy #20 started [history syncer #3]
170596:20221022:230546.315 proxy #26 started [poller #3]
170591:20221022:230546.315 proxy #21 started [history syncer #4]
170570:20221022:230546.315 received configuration data from server at "192.168.24.170", datalen 14792
170597:20221022:230546.316 proxy #27 started [poller #4]
170598:20221022:230546.318 proxy #28 started [poller #5]
170600:20221022:230546.318 proxy #30 started [unreachable poller #2]
170608:20221022:230546.321 proxy #38 started [icmp pinger #5]
170599:20221022:230546.322 proxy #29 started [unreachable poller #1]
170601:20221022:230546.322 proxy #31 started [unreachable poller #3]
170605:20221022:230546.322 proxy #35 started [icmp pinger #2]
170604:20221022:230546.322 proxy #34 started [icmp pinger #1]
170609:20221022:230546.322 proxy #39 started [availability manager #1]
170602:20221022:230546.323 proxy #32 started [unreachable poller #4]
170603:20221022:230546.325 proxy #33 started [unreachable poller #5]
170606:20221022:230546.326 proxy #36 started [icmp pinger #3]
170610:20221022:230546.326 proxy #40 started [odbc poller #1]
170607:20221022:230546.326 proxy #37 started [icmp pinger #4]
170582:20221022:230546.410 Unable to connect to [192.168.24.169]:10051 [cannot connect to [[192.168.24.169]:10051]: [111] Connection refused]
170572:20221022:230547.671 cannot send list of active checks to "127.0.0.1": host [test-zbx-proxy1] not found
170570:20221022:230726.392 received configuration data from server at "192.168.24.170", datalen 14792
170574:20221022:230748.671 cannot send list of active checks to "127.0.0.1": host [test-zbx-proxy1] not found
170570:20221022:230906.500 received configuration data from server at "192.168.24.170", datalen 14792
170575:20221022:230949.672 cannot send list of active checks to "127.0.0.1": host [test-zbx-proxy1] not found
170570:20221022:231046.627 received configuration data from server at "192.168.24.170", datalen 14792
170575:20221022:231150.671 cannot send list of active checks to "127.0.0.1": host [test-zbx-proxy1] not found
170570:20221022:231226.748 received configuration data from server at "192.168.24.170", datalen 14792
170572:20221022:231351.671 cannot send list of active checks to "127.0.0.1": host [test-zbx-proxy1] not found
170570:20221022:231406.859 received configuration data from server at "192.168.24.170", datalen 14792

5) If I set Interface: Agent to 127.0.0.1 in the previous case, the situation is still the same. Only "Active agent availability" has a value, and that is unknown (0).
Previously, I remembered that measuring yourself through the active agent went through the active proxy - but now it doesn't, maybe I remembered wrong.
6) Measuring a passive agent for itself through the active proxy also works.
In this case, the linux host setting:
host name: test-zbx-proxy1
template: Zabbix agent for Linux
Interface: Agent, 127.0.0.1
Monitored by proxy: test-zbx-proxy1

So essentially, only the passive agents method works, it goes directly from the server and also through an active proxy.

Code:

The test-zbx-srv1 (192.168.24.169) Zabbix zabbix_server.conf:
LogFile=/var/log/zabbix/zabbix_server.log
LogFileSize=0
PidFile=/run/zabbix/zabbix_server.pid
SocketDir=/run/zabbix
DBHost=192.168.24.174
DBName=tzabbixdb
DBUser=zabbix
DBPassword=password
DBPort=5000
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
Timeout=4
LogSlowQueries=3000
StatsAllowedIP=127.0.0.1
HANodeName=test-zbx-srv1
NodeAddress=192.168.24.169:10051

The test-zbx-srv2 (192.168.24.170) Zabbix zabbix_server.conf:
LogFile=/var/log/zabbix/zabbix_server.log
LogFileSize=0
PidFile=/run/zabbix/zabbix_server.pid
SocketDir=/run/zabbix
DBHost=192.168.24.174
DBName=tzabbixdb
DBUser=zabbix
DBPassword=password
DBPort=5000
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
Timeout=30
LogSlowQueries=3000
StatsAllowedIP=127.0.0.1
HANodeName=test-zbx-srv2
NodeAddress=192.168.24.170:10051

The test-zbx-proxy1 (192.168.24.178) zabbix_proxy.conf:
Server=192.168.24.169;192.168.24.170
Hostname=test-zbx-proxy1
LogFile=/var/log/zabbix/zabbix_proxy.log
LogFileSize=0
PidFile=/run/zabbix/zabbix_proxy.pid
SocketDir=/run/zabbix
DBName=zabbix_proxy
DBUser=zabbix
DBPassword=password
ProxyOfflineBuffer=48
ConfigFrequency=100
StartPollers=5
StartPollersUnreachable=5
StartTrappers=5
StartPingers=5
StartDiscoverers=2
StartHTTPPollers=2
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
CacheSize=128M
HistoryCacheSize=64M
HistoryIndexCacheSize=32M
Timeout=15
LogSlowQueries=3000
StatsAllowedIP=127.0.0.1

The test-zbx-fe2 (192.168.24.176) linux host zabbix_agent2.conf:
PidFile=/run/zabbix/zabbix_agent2.pid
LogFile=/var/log/zabbix/zabbix_agent2.log
LogFileSize=0
Server=127.0.0.1,192.168.24.169,192.168.24.170,192.168.24.178
ServerActive=127.0.0.1;192.168.24.169;192.168.24.170;192.168.24.178
Hostname=test-zbx-fe2
Include=/etc/zabbix/zabbix_agent2.d/*.conf
ControlSocket=/tmp/agent.sock
Include=./zabbix_agent2.d/plugins.d/*.conf

Do you have any other ideas as to what the problem might be?
I'm still thinking about replacing 6.2 with Zabbix version 6.0 everywhere, maybe then...

Ad Widget

Linux host with active agent isn't work

Linux host with active agent isn't work

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment