Php-fpm: Service is down

[User555]

Hi. This problem is periodically observed. It starts with the transition of one of the proxy groups to "Degrading" state, еhen everything goes as in the log, and at the same time messages arrive "Php-fpm: Service is down". The Zabbix web-interface becomes unavailable for about 3 minutes. Then everything is restored.

625592:20250922:101128.270 Proxy group "grpPRX-8" changed state from online to degrading
625592:20250922:101133.271 Proxy "ZBX-PRX-8" changed state from online to offline
625592:20250922:101133.271 Proxy group "grpPRX-8" changed state from degrading to offline
625592:20250922:101133.272 Proxy group "grpPRX-9" changed state from online to degrading
625503:20250922:101357.045 slow query: 192.117789 sec, "delete from history_uint where itemid=6158322 and clock<1757226641"
625592:20250922:101357.073 slow query: 142.799832 sec, "update proxy_group_rtdata set state=1 where proxy_groupid=2;.update proxy_rtdata set state=1 where proxyid=15760;."
625535:20250922:101357.075 slow query: 139.829399 sec, "update proxy_rtdata set lastaccess=1758525091 where proxyid=17825;.update proxy_rtdata set lastaccess=1758525097 where proxyid=17826;."
625513:20250922:101357.081 slow query: 191.904951 sec, "commit;"
625497:20250922:101357.083 slow query: 192.007378 sec, "commit;"
625510:20250922:101357.084 slow query: 189.810351 sec, "commit;"
625514:20250922:101357.084 slow query: 190.828205 sec, "commit;"
625517:20250922:101357.085 slow query: 190.814572 sec, "commit;"
625484:20250922:101357.086 slow query: 191.123112 sec, "commit;"
625486:20250922:101357.086 slow query: 29.899148 sec, "delete from changelog where clock<1758521607"
625484:20250922:101357.087 cannot write to IPC socket: Broken pipe
625511:20250922:101357.087 slow query: 191.787927 sec, "commit;"
625518:20250922:101357.088 slow query: 191.668914 sec, "commit;"
625484:20250922:101357.088 cannot write to IPC socket: Broken pipe
625484:20250922:101357.088 cannot write to IPC socket: Broken pipe
625527:20250922:101357.090 slow query: 108.668173 sec, "commit;"
625526:20250922:101357.090 slow query: 84.122823 sec, "commit;"
625515:20250922:101357.091 slow query: 190.828850 sec, "commit;"
625496:20250922:101357.096 slow query: 192.026826 sec, "commit;"
625525:20250922:101357.096 slow query: 190.810805 sec, "commit;"
625512:20250922:101357.096 slow query: 191.857310 sec, "commit;"
625537:20250922:101357.098 slow query: 187.924083 sec, "commit;"
625498:20250922:101357.099 slow query: 192.028322 sec, "commit;"
625516:20250922:101357.099 slow query: 189.789614 sec, "commit;"
625509:20250922:101357.100 slow query: 191.873495 sec, "commit;"
625519:20250922:101357.103 slow query: 191.890218 sec, "commit;"
625533:20250922:101357.103 slow query: 151.703729 sec, "update proxy_rtdata set lastaccess=1758525085 where proxyid=15759;.update proxy_rtdata set lastaccess=1758525079 where proxyid=15875;.update proxy_rtdata set lastaccess=1758525084 where proxyid=17825;.update proxy_rtdata set lastaccess=1758525084 where proxyid=17826;."
625572:20250922:101357.118 slow query: 192.057793 sec, "commit;"
625532:20250922:101357.118 slow query: 169.195197 sec, "update proxy_rtdata set lastaccess=1758525066 where proxyid=15758;.update proxy_rtdata set lastaccess=1758525066 where proxyid=15759;.update proxy_rtdata set lastaccess=1758525061 where proxyid=15760;.update proxy_rtdata set lastaccess=1758525067 where proxyid=15761;.update proxy_rtdata set lastaccess=1758525066 where proxyid=15875;.update proxy_rtdata set lastaccess=1758525066 where proxyid=17825;.update proxy_rtdata set lastaccess=1758525065 where proxyid=17826;."
625592:20250922:101357.125 Proxy "zbx-prx" changed state from online to offline
625592:20250922:101357.126 Proxy "ZBX-PRX-5" changed state from online to offline
625592:20250922:101357.126 Proxy "ZBX-PRX-9" changed state from online to offline
625592:20250922:101357.126 Proxy "ZBX-PRX-6" changed state from online to offline
625592:20250922:101357.126 Proxy "ZBX-PRX-7" changed state from online to offline
625592:20250922:101357.126 Proxy group "grpPRX-5" changed state from online to offline
625592:20250922:101357.126 Proxy group "grpPRX-9" changed state from degrading to offline
625592:20250922:101357.126 Proxy group "grpPRX-6" changed state from online to offline
625592:20250922:101357.126 Proxy group "grpPRX-7" changed state from online to offline
625531:20250922:101357.134 slow query: 145.274636 sec, "update proxy_rtdata set lastaccess=1758525085 where proxyid=15759;.update proxy_rtdata set lastaccess=1758525091 where proxyid=17825;.update proxy_rtdata set lastaccess=1758525090 where proxyid=17826;."
625540:20250922:101357.150 slow query: 181.766629 sec, "update proxy_rtdata set lastaccess=1758525055 where proxyid=15502;.update proxy_rtdata set lastaccess=1758525049 where proxyid=15750;.update proxy_rtdata set lastaccess=1758525054 where proxyid=15758;.update proxy_rtdata set lastaccess=1758525054 where proxyid=15759;.update proxy_rtdata set lastaccess=1758525054 where proxyid=15760;.update proxy_rtdata set lastaccess=1758525054 where proxyid=15761;.update proxy_rtdata set lastaccess=1758525054 where proxyid=15875;.update proxy_rtdata set lastaccess=1758525054 where proxyid=17825;.update proxy_rtdata set lastaccess=1758525054 where proxyid=17826;."
625536:20250922:101357.168 slow query: 164.011072 sec, "update proxy_rtdata set lastaccess=1758525073 where proxyid=15758;.update proxy_rtdata set lastaccess=1758525072 where proxyid=15759;.update proxy_rtdata set lastaccess=1758525067 where proxyid=15761;.update proxy_rtdata set lastaccess=1758525072 where proxyid=15875;.update proxy_rtdata set lastaccess=1758525072 where proxyid=17825;.update proxy_rtdata set lastaccess=1758525072 where proxyid=17826;."
625534:20250922:101357.213 slow query: 158.181761 sec, "update proxy_rtdata set lastaccess=1758525073 where proxyid=15758;.update proxy_rtdata set lastaccess=1758525078 where proxyid=15759;.update proxy_rtdata set lastaccess=1758525079 where proxyid=15875;.update proxy_rtdata set lastaccess=1758525077 where proxyid=17825;.update proxy_rtdata set lastaccess=1758525078 where proxyid=17826;."
625538:20250922:101357.230 slow query: 175.595575 sec, "update proxy_rtdata set lastaccess=1758525055 where proxyid=15502;.update proxy_rtdata set lastaccess=1758525060 where proxyid=15758;.update proxy_rtdata set lastaccess=1758525060 where proxyid=15759;.update proxy_rtdata set lastaccess=1758525061 where proxyid=15760;.update proxy_rtdata set lastaccess=1758525060 where proxyid=15761;.update proxy_rtdata set lastaccess=1758525060 where proxyid=15875;.update proxy_rtdata set lastaccess=1758525060 where proxyid=17825;.update proxy_rtdata set lastaccess=1758525060 where proxyid=17826;."
625592:20250922:101402.154 Proxy "zbx-prx" changed state from offline to online
625514:20250922:101404.700 item "ZBX-SRV:zabbix[cluster,discovery,nodes]" became not supported: Timeout while waiting for response
625510:20250922:101404.703 item "ZBX-SRV:web.page.get["{$PHP_FPM.HOST}","{$PHP_FPM.PING.PAGE}","{$PH P_FP M.PORT}"]" became not supported: Timeout occurred while gathering data.
625510:20250922:101404.703 item "ZBX-SRVhp-fpm.ping" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.726 item "ZBX-SRV:web.page.get["{$PHP_FPM.HOST}","{$PHP_FPM.STATUS.PAGE}?json ","{ $PHP_FPM.PORT}"]" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.726 item "ZBX-SRVhp-fpm.conn_accepted.rate" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.726 item "ZBX-SRVhp-fpm.listen_queue" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.726 item "ZBX-SRVhp-fpm.listen_queue_len" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.726 item "ZBX-SRVhp-fpm.listen_queue_max" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.726 item "ZBX-SRVhp-fpm.max_children" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.726 item "ZBX-SRVhp-fpm.name" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.726 item "ZBX-SRVhp-fpm.process_manager" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.727 item "ZBX-SRVhp-fpm.processes_active" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.727 item "ZBX-SRVhp-fpm.processes_idle" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.727 item "ZBX-SRVhp-fpm.processes_max_active" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.727 item "ZBX-SRVhp-fpm.processes_total" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.727 item "ZBX-SRVhp-fpm.slow_requests" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.727 item "ZBX-SRVhp-fpm.start_time" became not supported: Timeout occurred while gathering data.
625509:20250922:101404.727 item "ZBX-SRVhp-fpm.uptime" became not supported: Timeout occurred while gathering data.
625498:20250922:101412.926 discovery rule "ZBX-SRV:zabbix.nodes.discovery" became not supported: Timeout while waiting for response
625511:20250922:101417.977 item "ZBX-SRVhp-fpm.listen_queue_usage" became not supported: Cannot evaluate function: item "/ZBX-SRV/php-fpm.listen_queue" is not supported at "last(//php-fpm.listen_queue)/(last(//php-fpm.listen_queue_len)+(last(//php-fpm.listen_queue_len)=0))*100".
625514:20250922:101420.078 item "ZBX-SRV:web.page.get["{$PHP_FPM.HOST}","{$PHP_FPM.PING.PAGE}","{$PH P_FP M.PORT}"]" became supported
625514:20250922:101420.078 item "ZBX-SRVhp-fpm.ping" became supported
625516:20250922:101421.296 item "ZBX-SRV:web.page.get["{$PHP_FPM.HOST}","{$PHP_FPM.STATUS.PAGE}?json ","{ $PHP_FPM.PORT}"]" became supported
625516:20250922:101421.296 item "ZBX-SRVhp-fpm.listen_queue" became supported
625516:20250922:101421.296 item "ZBX-SRVhp-fpm.listen_queue_len" became supported
625516:20250922:101421.296 item "ZBX-SRVhp-fpm.listen_queue_max" became supported
625516:20250922:101421.296 item "ZBX-SRVhp-fpm.name" became supported
625516:20250922:101421.296 item "ZBX-SRVhp-fpm.process_manager" became supported
625516:20250922:101421.296 item "ZBX-SRVhp-fpm.processes_active" became supported
625516:20250922:101421.296 item "ZBX-SRVhp-fpm.processes_idle" became supported
625516:20250922:101421.297 item "ZBX-SRVhp-fpm.processes_max_active" became supported
625516:20250922:101421.297 item "ZBX-SRVhp-fpm.processes_total" became supported
625516:20250922:101421.297 item "ZBX-SRVhp-fpm.start_time" became supported
625516:20250922:101421.297 item "ZBX-SRVhp-fpm.uptime" became supported
625592:20250922:101427.170 Proxy "ZBX-PRX-9" changed state from offline to online
625592:20250922:101427.170 Proxy group "grpPRX-9" changed state from offline to recovering
625592:20250922:101432.184 Proxy "ZBX-PRX-5" changed state from offline to online
625592:20250922:101432.184 Proxy "ZBX-PRX-8" changed state from offline to online
625592:20250922:101432.184 Proxy "ZBX-PRX-7" changed state from offline to online
625592:20250922:101432.184 Proxy group "grpPRX-5" changed state from offline to recovering
625592:20250922:101432.184 Proxy group "grpPRX-8" changed state from offline to recovering
625592:20250922:101432.184 Proxy group "grpPRX-9" changed state from recovering to online
625592:20250922:101432.184 Proxy group "grpPRX-7" changed state from offline to recovering
625592:20250922:101437.200 Proxy "ZBX-PRX-6" changed state from offline to online
625592:20250922:101437.200 Proxy group "grpPRX-5" changed state from recovering to online
625592:20250922:101437.200 Proxy group "grpPRX-8" changed state from recovering to online
625592:20250922:101437.200 Proxy group "grpPRX-6" changed state from offline to recovering
625592:20250922:101437.200 Proxy group "grpPRX-7" changed state from recovering to online
625592:20250922:101442.217 Proxy group "grpPRX-6" changed state from recovering to online
625497:20250922:101443.115 discovery rule "ZBX-SRV:zabbix.nodes.discovery" became supported
625514:20250922:101517.577 item "ZBX-SRVhp-fpm.listen_queue_usage" became supported
625516:20250922:101521.164 item "ZBX-SRVhp-fpm.conn_accepted.rate" became supported
625516:20250922:101521.165 item "ZBX-SRVhp-fpm.max_children" became supported
625516:20250922:101521.165 item "ZBX-SRVhp-fpm.slow_requests" became supported

[User555]

Correction of the Topic name: Proxies switch from online to offline, the web interface becomes unavailable, and messages arrive "Php-fpm: Service is down", but php-fpm service is running

[cyber]

Please describe your setup also. Currently it seems like your DB is hanging there and everything just collapses because of that...
that php-fpm item just seems to be first in queue, that cannot be checked and then you get some timeouts and "not supported item" notification...

[User555]

The main problem is slow queries (about 180sec) and "cannot write to IPC socket: Broken pipe" . As I see it, this leads to about 3 minutes of unavailability of the web interface. A rotating ring is displayed.