Zabbix 5.4 stop Working

[Daniel4711]

Hello all,

unfortunately I need support with our Zabbix 5.4 server (Debian 10).
It has been running for about 5 weeks (I want to replace Icinga at my new employer) - Since tonight I have many error messages and Zabbix restarts constantly.

To our environment: We have about 300 hosts and 400 items per second.

The error message looks like this:
zabbix_server [17759]: ERROR [file and function: <lld_trigger.c,lld_expression_create>, revision:4c8f9aabe1, line:2518] Something impossible has just happened.



Zabbix Server Configuration:

LogFile=/var/log/zabbix/zabbix_server.log
LogFileSize=128
DebugLevel=3
PidFile=/run/zabbix/zabbix_server.pid
SocketDir=/run/zabbix
DBName=zabbix
DBUser=ZabbixDBUser
DBPassword=NotForYourEyes
StartPollers=25
StartPollersUnreachable=5
StartPingers=5
StartDiscoverers=10
StartVMwareCollectors=5
VMwareCacheSize=32M
VMwareTimeout=120
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
StartSNMPTrapper=1
CacheSize=256M
StartDBSyncers=2
TrendCacheSize=64M
TrendFunctionCacheSize=64M
ValueCacheSize=256M
Timeout=20
FpingLocation=/usr/bin/fping
Fping6Location=/usr/bin/fping6
LogSlowQueries=3000
StatsAllowedIP=127.0.0.1

The log file between starting and stopping Zabbix looks like this:

17678:20210714:055314.266 Starting Zabbix Server. Zabbix 5.4.2 (revision 4c8f9aabe1).
17678:20210714:055314.266 ****** Enabled features ******
17678:20210714:055314.266 SNMP monitoring: YES
17678:20210714:055314.266 IPMI monitoring: YES
17678:20210714:055314.266 Web monitoring: YES
17678:20210714:055314.266 VMware monitoring: YES
17678:20210714:055314.266 SMTP authentication: YES
17678:20210714:055314.266 ODBC: YES
17678:20210714:055314.266 SSH support: YES
17678:20210714:055314.266 IPv6 support: YES
17678:20210714:055314.266 TLS support: YES
17678:20210714:055314.266 ******************************
17678:20210714:055314.266 using configuration file: /etc/zabbix/zabbix_server.conf
17678:20210714:055314.271 current database version (mandatory/optional): 05040000/05040000
17678:20210714:055314.271 required mandatory version: 05040000
17678:20210714:055314.278 server #0 started [main process]
17679:20210714:055314.279 server #1 started [configuration syncer #1]
17681:20210714:055318.283 server #2 started [housekeeper #1]
17682:20210714:055318.284 server #3 started [timer #1]
17689:20210714:055318.285 server #10 started [discoverer #6]
17687:20210714:055318.286 server #8 started [discoverer #4]
17695:20210714:055318.290 server #16 started [history syncer #2]
17697:20210714:055318.291 server #18 started [snmp trapper #1]
17699:20210714:055318.291 server #20 started [self-monitoring #1]
17701:20210714:055318.291 server #22 started [vmware collector #2]
17693:20210714:055318.292 server #14 started [discoverer #10]
17685:20210714:055318.294 server #6 started [discoverer #2]
17690:20210714:055318.298 server #11 started [discoverer #7]
17692:20210714:055318.299 server #13 started [discoverer #9]
17703:20210714:055318.301 server #24 started [vmware collector #4]
17694:20210714:055318.301 server #15 started [history syncer #1]
17691:20210714:055318.301 server #12 started [discoverer #8]
17688:20210714:055318.307 server #9 started [discoverer #5]
17704:20210714:055318.307 server #25 started [vmware collector #5]
17696:20210714:055318.307 server #17 started [escalator #1]
17698:20210714:055318.309 server #19 started [proxy poller #1]
17705:20210714:055318.309 server #26 started [task manager #1]
17707:20210714:055318.310 server #28 started [poller #2]
17700:20210714:055318.311 server #21 started [vmware collector #1]
17702:20210714:055318.312 server #23 started [vmware collector #3]
17706:20210714:055318.312 server #27 started [poller #1]
17711:20210714:055318.315 server #32 started [poller #6]
17708:20210714:055318.315 server #29 started [poller #3]
17713:20210714:055318.319 server #34 started [poller #8]
17709:20210714:055318.319 server #30 started [poller #4]
17710:20210714:055318.322 server #31 started [poller #5]
17686:20210714:055318.327 server #7 started [discoverer #3]
17715:20210714:055318.330 server #36 started [poller #10]
17712:20210714:055318.330 server #33 started [poller #7]
17684:20210714:055318.335 server #5 started [discoverer #1]
17714:20210714:055318.343 server #35 started [poller #9]
17683:20210714:055318.359 server #4 started [http poller #1]
17720:20210714:055318.363 server #41 started [poller #15]
17722:20210714:055318.379 server #43 started [poller #17]
17717:20210714:055318.383 server #38 started [poller #12]
17718:20210714:055318.387 server #39 started [poller #13]
17719:20210714:055318.390 server #40 started [poller #14]
17721:20210714:055318.394 server #42 started [poller #16]
17724:20210714:055318.394 server #45 started [poller #19]
17716:20210714:055318.399 server #37 started [poller #11]
17723:20210714:055318.403 server #44 started [poller #18]
17727:20210714:055318.422 server #48 started [poller #22]
17729:20210714:055318.427 server #50 started [poller #24]
17728:20210714:055318.435 server #49 started [poller #23]
17726:20210714:055318.439 server #47 started [poller #21]
17731:20210714:055318.451 server #52 started [unreachable poller #1]
17730:20210714:055318.463 server #51 started [poller #25]
17733:20210714:055318.467 server #54 started [unreachable poller #3]
17732:20210714:055318.471 server #53 started [unreachable poller #2]
17725:20210714:055318.479 server #46 started [poller #20]
17735:20210714:055318.502 server #56 started [unreachable poller #5]
17736:20210714:055318.504 server #57 started [trapper #1]
17737:20210714:055318.507 server #58 started [trapper #2]
17738:20210714:055318.515 server #59 started [trapper #3]
17739:20210714:055318.519 server #60 started [trapper #4]
17740:20210714:055318.535 server #61 started [trapper #5]
17741:20210714:055318.555 server #62 started [icmp pinger #1]
17734:20210714:055318.555 server #55 started [unreachable poller #4]
17744:20210714:055318.607 server #64 started [icmp pinger #3]
17745:20210714:055318.623 server #65 started [icmp pinger #4]
17746:20210714:055318.623 server #66 started [icmp pinger #5]
17747:20210714:055318.627 server #67 started [alert manager #1]
17749:20210714:055318.629 server #69 started [alerter #2]
17748:20210714:055318.631 server #68 started [alerter #1]
17750:20210714:055318.634 server #70 started [alerter #3]
17743:20210714:055318.635 server #63 started [icmp pinger #2]
17751:20210714:055318.635 server #71 started [preprocessing manager #1]
17756:20210714:055318.660 server #74 started [preprocessing worker #3]
17758:20210714:055318.661 server #76 started [lld worker #1]
17757:20210714:055318.667 server #75 started [lld manager #1]
17755:20210714:055318.668 server #73 started [preprocessing worker #2]
17759:20210714:055318.671 server #77 started [lld worker #2]
17761:20210714:055318.679 server #79 started [history poller #1]
17760:20210714:055318.679 server #78 started [alert syncer #1]
17762:20210714:055318.687 server #80 started [history poller #2]
17754:20210714:055318.687 server #72 started [preprocessing worker #1]
17766:20210714:055318.698 server #82 started [history poller #4]
17765:20210714:055318.702 server #81 started [history poller #3]
17767:20210714:055318.702 server #83 started [history poller #5]
17768:20210714:055318.704 server #84 started [availability manager #1]
zabbix_server [17759]: ERROR [file and function: <lld_trigger.c,lld_expression_create>, revision:4c8f9aabe1, line:2518] Something impossible has just happened.
17759:20210714:055337.475 === Backtrace: ===
17759:20210714:055337.476 12: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](zbx_backtrace+0x3f) [0x555f31299fe6]
17759:20210714:055337.476 11: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](+0x168707) [0x555f311ed707]
17759:20210714:055337.476 10: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](+0x169215) [0x555f311ee215]
17759:20210714:055337.476 9: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](lld_update_triggers+0x2c2) [0x555f311f0ad1]
17759:20210714:055337.476 8: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](lld_process_discovery_rule+0x4f8) [0x555f311c9d45]
17759:20210714:055337.476 7: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](+0x141a37) [0x555f311c6a37]
17759:20210714:055337.476 6: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](lld_worker_thread+0x2f6) [0x555f311c7098]
17759:20210714:055337.476 5: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](zbx_thread_start+0x37) [0x555f312aa1d4]
17759:20210714:055337.476 4: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](MAIN_ZABBIX_ENTRY+0x1129) [0x555f310da00c]
17759:20210714:055337.476 3: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](daemon_start+0x2ff) [0x555f31299bf2]
17759:20210714:055337.476 2: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](main+0x2f5) [0x555f310d8e0d]
17759:20210714:055337.476 1: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7fb3bccd109b]
17759:20210714:055337.476 0: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](_start+0x2a) [0x555f310d7cea]
zabbix_server [17759]: ERROR [file and function: <lld_trigger.c,lld_expression_create>, revision:4c8f9aabe1, line:2518] Something impossible has just happened.
17759:20210714:055337.477 === Backtrace: ===
17759:20210714:055337.478 12: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](zbx_backtrace+0x3f) [0x555f31299fe6]
17759:20210714:055337.478 11: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](+0x168707) [0x555f311ed707]
17759:20210714:055337.478 10: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](+0x169278) [0x555f311ee278]
17759:20210714:055337.478 9: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](lld_update_triggers+0x2c2) [0x555f311f0ad1]
17759:20210714:055337.478 8: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](lld_process_discovery_rule+0x4f8) [0x555f311c9d45]
17759:20210714:055337.478 7: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](+0x141a37) [0x555f311c6a37]
17759:20210714:055337.478 6: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](lld_worker_thread+0x2f6) [0x555f311c7098]
17759:20210714:055337.478 5: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](zbx_thread_start+0x37) [0x555f312aa1d4]
17759:20210714:055337.478 4: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](MAIN_ZABBIX_ENTRY+0x1129) [0x555f310da00c]
17759:20210714:055337.478 3: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](daemon_start+0x2ff) [0x555f31299bf2]
17759:20210714:055337.478 2: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](main+0x2f5) [0x555f310d8e0d]
17759:20210714:055337.478 1: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7fb3bccd109b]
17759:20210714:055337.478 0: /usr/sbin/zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec](_start+0x2a) [0x555f310d7cea]
17759:20210714:055337.478 ERROR: rollback without transaction. Please report it to Zabbix Team.
zabbix_server: lld worker #2 [processed 2 LLD rules, idle 6.354767 sec during 6.371148 sec]: db.c:1101: zbx_db_rollback: Assertion `0' failed.
17678:20210714:055337.479 One child process died (PID:17759,exitcode/signal:6). Exiting ...
zabbix_server [17678]: Error waiting for process with PID 17759: [10] No child processes
17678:20210714:055337.518 syncing history data...
17678:20210714:055337.547 syncing history data... 100.000000%
17678:20210714:055337.547 syncing history data done
17678:20210714:055337.547 syncing trend data...
17678:20210714:055337.724 syncing trend data done
17678:20210714:055337.725 Zabbix Server stopped. Zabbix 5.4.2 (revision 4c8f9aabe1).
17845:20210714:055347.999 Starting Zabbix Server. Zabbix 5.4.2 (revision 4c8f9aabe1).
17845:20210714:055347.999 ****** Enabled features ******
17845:20210714:055347.999 SNMP monitoring: YES
17845:20210714:055347.999 IPMI monitoring: YES
17845:20210714:055347.999 Web monitoring: YES
17845:20210714:055347.999 VMware monitoring: YES
17845:20210714:055347.999 SMTP authentication: YES
17845:20210714:055347.999 ODBC: YES
17845:20210714:055347.999 SSH support: YES
17845:20210714:055347.999 IPv6 support: YES
17845:20210714:055348.000 TLS support: YES
17845:20210714:055348.000 ******************************
17845:20210714:055348.000 using configuration file: /etc/zabbix/zabbix_server.conf
17845:20210714:055348.005 current database version (mandatory/optional): 05040000/05040000
17845:20210714:055348.005 required mandatory version: 05040000
17845:20210714:055348.012 server #0 started [main process]
17846:20210714:055348.012 server #1 started [configuration syncer #1]
17848:20210714:055352.117 server #3 started [timer #1]
17850:20210714:055352.119 server #5 started [discoverer #1]
17852:20210714:055352.122 server #7 started [discoverer #3]
17860:20210714:055352.126 server #15 started [history syncer #1]
17862:20210714:055352.127 server #17 started [escalator #1]
17858:20210714:055352.128 server #13 started [discoverer #9]
17853:20210714:055352.129 server #8 started [discoverer #4]
17855:20210714:055352.130 server #10 started [discoverer #6]
17864:20210714:055352.131 server #19 started [proxy poller #1]
17856:20210714:055352.133 server #11 started [discoverer #7]
17857:20210714:055352.133 server #12 started [discoverer #8]
17859:20210714:055352.138 server #14 started [discoverer #10]
17866:20210714:055352.139 server #21 started [vmware collector #1]
17851:20210714:055352.139 server #6 started [discoverer #2]
17854:20210714:055352.142 server #9 started [discoverer #5]
17861:20210714:055352.143 server #16 started [history syncer #2]
17863:20210714:055352.143 server #18 started [snmp trapper #1]
17869:20210714:055352.144 server #24 started [vmware collector #4]
17865:20210714:055352.144 server #20 started [self-monitoring #1]
17867:20210714:055352.144 server #22 started [vmware collector #2]
17873:20210714:055352.145 server #28 started [poller #2]
17871:20210714:055352.147 server #26 started [task manager #1]
17875:20210714:055352.147 server #30 started [poller #4]
17870:20210714:055352.147 server #25 started [vmware collector #5]
17868:20210714:055352.147 server #23 started [vmware collector #3]
17872:20210714:055352.148 server #27 started [poller #1]
17849:20210714:055352.150 server #4 started [http poller #1]
17874:20210714:055352.152 server #29 started [poller #3]
17847:20210714:055352.163 server #2 started [housekeeper #1]
17877:20210714:055352.163 server #32 started [poller #6]
17879:20210714:055352.166 server #34 started [poller #8]
17878:20210714:055352.171 server #33 started [poller #7]
17880:20210714:055352.175 server #35 started [poller #9]
17882:20210714:055352.183 server #37 started [poller #11]
17876:20210714:055352.186 server #31 started [poller #5]
17881:20210714:055352.191 server #36 started [poller #10]
17884:20210714:055352.194 server #39 started [poller #13]
17883:20210714:055352.199 server #38 started [poller #12]
17887:20210714:055352.215 server #42 started [poller #16]
17888:20210714:055352.215 server #43 started [poller #17]
17886:20210714:055352.219 server #41 started [poller #15]
17889:20210714:055352.227 server #44 started [poller #18]
17890:20210714:055352.231 server #45 started [poller #19]
17892:20210714:055352.250 server #47 started [poller #21]
17891:20210714:055352.254 server #46 started [poller #20]
17894:20210714:055352.259 server #49 started [poller #23]
17893:20210714:055352.263 server #48 started [poller #22]
17885:20210714:055352.267 server #40 started [poller #14]
17896:20210714:055352.303 server #51 started [poller #25]
17897:20210714:055352.307 server #52 started [unreachable poller #1]
17898:20210714:055352.314 server #53 started [unreachable poller #2]
17899:20210714:055352.319 server #54 started [unreachable poller #3]
17900:20210714:055352.327 server #55 started [unreachable poller #4]
17901:20210714:055352.335 server #56 started [unreachable poller #5]
17902:20210714:055352.342 server #57 started [trapper #1]
17895:20210714:055352.361 server #50 started [poller #24]
17906:20210714:055352.379 server #61 started [trapper #5]
17904:20210714:055352.382 server #59 started [trapper #3]
17905:20210714:055352.388 server #60 started [trapper #4]
17907:20210714:055352.403 server #62 started [icmp pinger #1]
17908:20210714:055352.411 server #63 started [icmp pinger #2]
17909:20210714:055352.423 server #64 started [icmp pinger #3]
17903:20210714:055352.423 server #58 started [trapper #2]
17910:20210714:055352.424 server #65 started [icmp pinger #4]
17912:20210714:055352.431 server #67 started [alert manager #1]
17911:20210714:055352.435 server #66 started [icmp pinger #5]
17913:20210714:055352.441 server #68 started [alerter #1]
17915:20210714:055352.480 server #70 started [alerter #3]
17916:20210714:055352.483 server #71 started [preprocessing manager #1]
17917:20210714:055352.484 server #72 started [preprocessing worker #1]
17919:20210714:055352.495 server #74 started [preprocessing worker #3]
17920:20210714:055352.495 server #75 started [lld manager #1]
17918:20210714:055352.495 server #73 started [preprocessing worker #2]
17921:20210714:055352.496 server #76 started [lld worker #1]
17922:20210714:055352.501 server #77 started [lld worker #2]
17923:20210714:055352.507 server #78 started [alert syncer #1]
17914:20210714:055352.513 server #69 started [alerter #2]
17927:20210714:055352.534 server #80 started [history poller #2]
17928:20210714:055352.536 server #81 started [history poller #3]
17929:20210714:055352.537 server #82 started [history poller #4]
17930:20210714:055352.537 server #83 started [history poller #5]
17931:20210714:055352.538 server #84 started [availability manager #1]
17926:20210714:055352.550 server #79 started [history poller #1]
zabbix_server [17922]: ERROR [file and function: <lld_trigger.c,lld_expression_create>, revision:4c8f9aabe1, line:2518] Something impossible has just happened.
17922:20210714:055405.785 === Backtrace: ===
17922:20210714:055405.786 12: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](zbx_backtrace+0x3f) [0x5575377d2fe6]
17922:20210714:055405.786 11: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](+0x168707) [0x557537726707]
17922:20210714:055405.786 10: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](+0x169215) [0x557537727215]
17922:20210714:055405.786 9: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](lld_update_triggers+0x2c2) [0x557537729ad1]
17922:20210714:055405.786 8: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](lld_process_discovery_rule+0x4f8) [0x557537702d45]
17922:20210714:055405.786 7: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](+0x141a37) [0x5575376ffa37]
17922:20210714:055405.786 6: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](lld_worker_thread+0x2f6) [0x557537700098]
17922:20210714:055405.786 5: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](zbx_thread_start+0x37) [0x5575377e31d4]
17922:20210714:055405.786 4: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](MAIN_ZABBIX_ENTRY+0x1129) [0x55753761300c]
17922:20210714:055405.786 3: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](daemon_start+0x2ff) [0x5575377d2bf2]
17922:20210714:055405.786 2: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](main+0x2f5) [0x557537611e0d]
17922:20210714:055405.786 1: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f07aebba09b]
17922:20210714:055405.786 0: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](_start+0x2a) [0x557537610cea]
zabbix_server [17922]: ERROR [file and function: <lld_trigger.c,lld_expression_create>, revision:4c8f9aabe1, line:2518] Something impossible has just happened.
17922:20210714:055405.787 === Backtrace: ===
17922:20210714:055405.787 12: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](zbx_backtrace+0x3f) [0x5575377d2fe6]
17922:20210714:055405.787 11: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](+0x168707) [0x557537726707]
17922:20210714:055405.787 10: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](+0x169278) [0x557537727278]
17922:20210714:055405.787 9: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](lld_update_triggers+0x2c2) [0x557537729ad1]
17922:20210714:055405.787 8: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](lld_process_discovery_rule+0x4f8) [0x557537702d45]
17922:20210714:055405.787 7: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](+0x141a37) [0x5575376ffa37]
17922:20210714:055405.787 6: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](lld_worker_thread+0x2f6) [0x557537700098]
17922:20210714:055405.787 5: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](zbx_thread_start+0x37) [0x5575377e31d4]
17922:20210714:055405.787 4: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](MAIN_ZABBIX_ENTRY+0x1129) [0x55753761300c]
17922:20210714:055405.787 3: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](daemon_start+0x2ff) [0x5575377d2bf2]
17922:20210714:055405.787 2: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](main+0x2f5) [0x557537611e0d]
17922:20210714:055405.787 1: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f07aebba09b]
17922:20210714:055405.787 0: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec](_start+0x2a) [0x557537610cea]
17922:20210714:055405.787 ERROR: rollback without transaction. Please report it to Zabbix Team.
zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.171612 sec during 8.185259 sec]: db.c:1101: zbx_db_rollback: Assertion `0' failed.
17845:20210714:055405.788 One child process died (PID:17922,exitcode/signal:6). Exiting ...
zabbix_server [17845]: Error waiting for process with PID 17922: [10] No child processes
17845:20210714:055405.825 syncing history data...
17845:20210714:055405.854 syncing history data... 100.000000%
17845:20210714:055405.854 syncing history data done
17845:20210714:055405.854 syncing trend data...
17845:20210714:055405.984 syncing trend data done
17845:20210714:055405.984 Zabbix Server stopped. Zabbix 5.4.2 (revision 4c8f9aabe1).
17987:20210714:055416.251 Starting Zabbix Server. Zabbix 5.4.2 (revision 4c8f9aabe1).
17987:20210714:055416.251 ****** Enabled features ******
17987:20210714:055416.251 SNMP monitoring: YES
17987:20210714:055416.251 IPMI monitoring: YES
17987:20210714:055416.251 Web monitoring: YES
17987:20210714:055416.251 VMware monitoring: YES
17987:20210714:055416.251 SMTP authentication: YES
17987:20210714:055416.251 ODBC: YES
17987:20210714:055416.251 SSH support: YES
17987:20210714:055416.251 IPv6 support: YES
17987:20210714:055416.251 TLS support: YES
17987:20210714:055416.251 ******************************
17987:20210714:055416.251 using configuration file: /etc/zabbix/zabbix_server.conf
17987:20210714:055416.257 current database version (mandatory/optional): 05040000/05040000
17987:20210714:055416.257 required mandatory version: 05040000
17987:20210714:055416.263 server #0 started [main process]
17988:20210714:055416.264 server #1 started [configuration syncer #1]
17989:20210714:055420.268 server #2 started [housekeeper #1]
17991:20210714:055420.269 server #4 started [http poller #1]
17993:20210714:055420.270 server #6 started [discoverer #2]
17994:20210714:055420.275 server #7 started [discoverer #3]
17996:20210714:055420.279 server #9 started [discoverer #5]
18007:20210714:055420.279 server #20 started [self-monitoring #1]
18003:20210714:055420.279 server #16 started [history syncer #2]
18005:20210714:055420.279 server #18 started [snmp trapper #1]
18001:20210714:055420.279 server #14 started [discoverer #10]
17999:20210714:055420.281 server #12 started [discoverer #8]
17995:20210714:055420.283 server #8 started [discoverer #4]
17998:20210714:055420.287 server #11 started [discoverer #7]
18002:20210714:055420.287 server #15 started [history syncer #1]
18000:20210714:055420.287 server #13 started [discoverer #9]
17997:20210714:055420.291 server #10 started [discoverer #6]
18004:20210714:055420.291 server #17 started [escalator #1]
18006:20210714:055420.293 server #19 started [proxy poller #1]
18009:20210714:055420.293 server #22 started [vmware collector #2]
18008:20210714:055420.294 server #21 started [vmware collector #1]
18011:20210714:055420.294 server #24 started [vmware collector #4]
18015:20210714:055420.295 server #28 started [poller #2]
18012:20210714:055420.295 server #25 started [vmware collector #5]
18010:20210714:055420.295 server #23 started [vmware collector #3]
18016:20210714:055420.296 server #29 started [poller #3]
17992:20210714:055420.299 server #5 started [discoverer #1]
18014:20210714:055420.299 server #27 started [poller #1]
18017:20210714:055420.303 server #30 started [poller #4]
18018:20210714:055420.305 server #31 started [poller #5]
18013:20210714:055420.306 server #26 started [task manager #1]
18019:20210714:055420.307 server #32 started [poller #6]
18020:20210714:055420.315 server #33 started [poller #7]
17990:20210714:055420.331 server #3 started [timer #1]
18023:20210714:055420.331 server #36 started [poller #10]
18022:20210714:055420.332 server #35 started [poller #9]
18025:20210714:055420.334 server #38 started [poller #12]
18024:20210714:055420.335 server #37 started [poller #11]
18027:20210714:055420.351 server #40 started [poller #14]
18026:20210714:055420.354 server #39 started [poller #13]
18021:20210714:055420.359 server #34 started [poller #8]
18028:20210714:055420.363 server #41 started [poller #15]
18031:20210714:055420.391 server #44 started [poller #18]
18030:20210714:055420.391 server #43 started [poller #17]
18032:20210714:055420.399 server #45 started [poller #19]
18033:20210714:055420.403 server #46 started [poller #20]
18034:20210714:055420.411 server #47 started [poller #21]
18035:20210714:055420.427 server #48 started [poller #22]
18029:20210714:055420.427 server #42 started [poller #16]
18037:20210714:055420.467 server #50 started [poller #24]
18039:20210714:055420.471 server #52 started [unreachable poller #1]
18038:20210714:055420.475 server #51 started [poller #25]
18040:20210714:055420.479 server #53 started [unreachable poller #2]
18041:20210714:055420.487 server #54 started [unreachable poller #3]
18042:20210714:055420.498 server #55 started [unreachable poller #4]
18043:20210714:055420.502 server #56 started [unreachable poller #5]
18036:20210714:055420.511 server #49 started [poller #23]
18045:20210714:055420.515 server #58 started [trapper #2]
18044:20210714:055420.516 server #57 started [trapper #1]
18046:20210714:055420.519 server #59 started [trapper #3]
18047:20210714:055420.522 server #60 started [trapper #4]
18048:20210714:055420.531 server #61 started [trapper #5]
18049:20210714:055420.531 server #62 started [icmp pinger #1]
18051:20210714:055420.551 server #64 started [icmp pinger #3]
18050:20210714:055420.559 server #63 started [icmp pinger #2]
18052:20210714:055420.573 server #65 started [icmp pinger #4]
18054:20210714:055420.580 server #67 started [alert manager #1]
18053:20210714:055420.583 server #66 started [icmp pinger #5]
18055:20210714:055420.599 server #68 started [alerter #1]
18059:20210714:055420.633 server #69 started [alerter #2]
18060:20210714:055420.633 server #70 started [alerter #3]
18061:20210714:055420.634 server #71 started [preprocessing manager #1]
18062:20210714:055420.643 server #72 started [preprocessing worker #1]
18064:20210714:055420.649 server #74 started [preprocessing worker #3]
18063:20210714:055420.650 server #73 started [preprocessing worker #2]
18069:20210714:055420.671 server #77 started [lld worker #2]
18068:20210714:055420.674 server #76 started [lld worker #1]
18070:20210714:055420.675 server #78 started [alert syncer #1]
18071:20210714:055420.681 server #79 started [history poller #1]
18072:20210714:055420.685 server #80 started [history poller #2]
18073:20210714:055420.691 server #81 started [history poller #3]
18067:20210714:055420.692 server #75 started [lld manager #1]
18074:20210714:055420.695 server #82 started [history poller #4]
18079:20210714:055420.712 server #84 started [availability manager #1]
18078:20210714:055420.719 server #83 started [history poller #5]
17987:20210714:055427.000 Got signal [signal:15(SIGTERM),sender_pid:18132,sender_uid:0,r eason:0]. Exiting ...
18002:20210714:055427.000 syncing history data in progress...
18050:20210714:055427.004 fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18052:20210714:055427.005 fping failed: 172.19.16.59 : [1], 84 bytes, 1.05 ms (1.26 avg, 0% loss)
18002:20210714:055427.012 item "Serverraum Elb.:icmppingsec" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.012 item "LAN3HalleBA1_2:icmppingsec" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.012 item "LAN3HalleBA1_2:icmppingloss" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.012 item "LAN3HalleBA1_2:icmpping" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.012 item "Vertrieb:icmppingloss" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.012 item "Vertrieb:icmppingsec" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.012 item "Vertrieb:icmpping" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.012 item "LANBA2_3:icmpping" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.012 item "LANBA2_3:icmppingsec" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.012 item "LANBA2_3:icmppingloss" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.012 item "Serverraum Elb.:icmpping" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.012 item "Serverraum Elb.:icmppingloss" became not supported: fping failed: 172.19.16.60 : [0], 84 bytes, 0.67 ms (0.67 avg, 0% loss)
18002:20210714:055427.016 syncing history data... 100.000000%
18002:20210714:055427.016 syncing history data done
17987:20210714:055427.756 syncing trend data...
17987:20210714:055427.780 syncing trend data done
17987:20210714:055427.780 Zabbix Server stopped. Zabbix 5.4.2 (revision 4c8f9aabe1).

I hope someone has a tip for me!

[perqvind]

If you are using Mysql then I would check the lock status, you can use this command SHOW OPEN TABLES to view open and locked tables, also see if the CPU is running high.

[Daniel4711]

Hello and thank you for your answer.
I have found the error.
I have created a trigger in the Vmware Discovery Template.
This caused the server to crash... for whatever reason.

19135:20210714:133343.253 In lld_expression_create() expression:'{1}>{$FREEDISKSPACEPERCENT:"C:\"}'
19135:20210714:133343.253 Invalid trigger "Free Space on {$VMWARE.URL} - Disk: C:\ below {$FREEDISKSPACEPERCENT}%" recovery expression: invalid token starting with "{$FREEDISKSPACEPERCENT:"C:\"}"
zabbix_server [19135]: ERROR [file and function: <lld_trigger.c,lld_expression_create>, revision:4c8f9aabe1, line:2518] Something impossible has just happened.
19135:20210714:133343.253 === Backtrace: ===
19135:20210714:133343.253 12: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](zbx_backtrace+0x3f) [0x562262940fe6]
19135:20210714:133343.253 11: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](+0x168707) [0x562262894707]
19135:20210714:133343.253 10: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](+0x169278) [0x562262895278]
19135:20210714:133343.253 9: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](lld_update_triggers+0x2c2) [0x562262897ad1]
19135:20210714:133343.253 8: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](lld_process_discovery_rule+0x4f8) [0x562262870d45]
19135:20210714:133343.253 7: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](+0x141a37) [0x56226286da37]
19135:20210714:133343.253 6: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](lld_worker_thread+0x2f6) [0x56226286e098]
19135:20210714:133343.253 5: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](zbx_thread_start+0x37) [0x5622629511d4]
19135:20210714:133343.253 4: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](MAIN_ZABBIX_ENTRY+0x1129) [0x56226278100c]
19135:20210714:133343.253 3: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](daemon_start+0x2ff) [0x562262940bf2]
19135:20210714:133343.253 2: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](main+0x2f5) [0x56226277fe0d]
19135:20210714:133343.253 1: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7fe0a8fa609b]
19135:20210714:133343.253 0: /usr/sbin/zabbix_server: lld worker #2 [processed 1 LLD rules, idle 8.204005 sec during 8.256361 sec](_start+0x2a) [0x56226277ecea]
19135:20210714:133343.254 End of lld_expression_create() expression:'{1}>{$FREEDISKSPACEPERCENT:"C:\"}'
19135:20210714:133343.254 ERROR: rollback without transaction. Please report it to Zabbix Team.