Ad Widget

**tim.mooney** · 03-02-2025, 22:21

The zabbix_server process should be writing some logs, probably under /var/log/zabbix/ in its own file. I would start by looking at that file to see if there are any clues there. It's possible to increase the verbosity of log messages to that file, either at runtime using the -R option to zabbix_server or by changing the logging parameters in the server configuration file and restarting.

In addition to the messages that zabbix_server writes, systemd may have relevant messages logged about why it's crashing. I typically would use

Code:

sudo systemctl status -l zabbix-server.service

to review the systemd messages, but some people prefer "journalctl" instead.

**nicolasfo** · 04-02-2025, 08:10

Hello,
New crash this morning, seems at 3:00, I read the zabbix logs, here's what I have :

939:20250204:024208.509 slow query: 9.977608 sec, "insert into history_uint (itemid,clock,ns,value) values (51897,1738633317,994279242,1074525);."
930:20250204:024208.513 slow query: 9.913071 sec, "insert into history_uint (itemid,clock,ns,value) values (55882,1738633315,696495767,1),(64558,1738633318,5 51704876,2550136832),(71632,1738633317,682353409,0 ),(71634,1738633317,681955104,200);."
945:20250204:024208.513 slow query: 10.057466 sec, "insert into history_uint (itemid,clock,ns,value) values (32037,1738633317,592733672,1),(32217,1738633317,5 92849944,0),(33837,1738633317,592689108,0),(33897, 1738633317,593919604,0),(50109,1738633318,51563563 ,1),(64617,1738633317,596421693,0),(75657,17386333 17,610170736,4),(75717,1738633317,597625608,806486 016);."
901:20250204:024208.532 slow query: 5.056954 sec, "insert into history_uint (itemid,clock,ns,value) values (32162,1738633322,142869259,0),(32222,1738633322,5 93532797,0),(32822,1738633322,567628773,1118130175 ),(32882,1738633322,142951461,3686),(32942,1738633 322,142914960,0),(33002,1738633322,142009440,0),(3 3062,1738633322,142592853,255),(33662,1738633322,1 42555152,0),(33842,1738633322,593192338,0),(33902, 1738633322,594563081,0),(33962,1738633322,14205234 1,255),(34022,1738633322,142414749,255),(34322,173 8633322,592996494,1738633333),(38342,1738633322,22 5644970,1738633322),(38431,1738633321,957851164,30 36983),(38521,1738633321,964251695,0),(39781,17386 33321,729435238,0),(40201,1738633321,700938414,343 59738368),(40441,1738633321,700908113,12293120),(4 0501,1738633321,706644499,0),(48002,1738633322,228 104969,108),(48840,1738633320,497710753,1),(48851, 1738633321,133184453,1),(49322,1738633322,60002300 1,0),(50884,1738633322,316188073,34491444),(51901, 1738633322,3490174,4),(52892,1738633322,142196144, 3031303),(54842,1738633322,226393769,4075),(55142, 1738633322,225093070,8588910592),(60962,1738633322 ,567633214,22),(64081,1738633321,958263240,1),(644 42,1738633322,145730023,0),(64652,1738633322,14247 0750,3823214),(65882,1738633322,176464388,0),(6714 3,1738633323,106444457,1),(70202,1738633322,147957 072,6),(70622,1738633322,593902813,1),(70681,17386 33321,957556384,2708418560),(71736,1738633321,8754 02545,0),(71738,1738633321,875038256,200),(75282,1 738633322,158695912,2);."
924:20250204:024208.542 slow query: 6.068259 sec, "insert into history_uint (itemid,clock,ns,value) values (32161,1738633321,140400092,0),(32221,1738633321,5 94368928,0),(32941,1738633321,141200310,0),(33001, 1738633321,140518295,0),(33061,1738633321,14036329 1,255),(33661,1738633321,140162387,0),(33841,17386 33321,594257393,0),(33901,1738633321,595303419,0), (33961,1738633321,140198588,255),(34021,1738633321 ,141402314,255),(38040,1738633320,957339188,164971 92),(39780,1738633320,741392930,0),(40320,17386333 20,702967758,12293910),(40440,1738633320,703299962 ,130170880),(40500,1738633320,708211436,0),(44730, 1738633319,371434736,1),(48001,1738633321,22314484 9,993),(48871,1738633322,34570337,12446258),(49321 ,1738633321,598590744,1),(50761,1738633321,1605158 34,0),(53131,1738633321,139985283,3865944),(54781, 1738633321,225457349,1),(54841,1738633321,23942634 6,245),(55980,1738633320,704740084,16002981036032) ,(56070,1738633320,703839371,2748177),(60251,17386 33321,107287713,0),(60252,1738633321,107287713,0), (60253,1738633321,107287713,0),(60254,1738633321,1 07287713,0),(60255,1738633321,107287713,0),(60256, 1738633321,107287713,0),(60257,1738633321,10728771 3,0),(60258,1738633321,107287713,0),(62468,1738633 321,94148370,0),(62469,1738633321,94148370,0),(624 70,1738633321,94148370,0),(62471,1738633321,941483 70,0),(62472,1738633321,94148370,0),(62473,1738633 321,94148370,0),(62474,1738633321,94148370,0),(624 75,1738633321,94148370,0),(64441,1738633321,144144 076,0),(65881,1738633321,177713906,0),(67102,17386 33321,107287713,0),(67112,1738633321,94148370,0),( 67261,1738633321,593820785,8589934592),(69960,1738 633320,958228016,128786432),(72001,1738633321,1396 72676,0),(73501,1738633321,75324976,1),(74088,1738 633321,90359267,0),(74089,1738633321,90359267,0),( 74090,1738633321,90359267,0),(74091,1738633321,903 59267,0),(74092,1738633321,90359267,0),(74093,1738 633321,90359267,0),(74094,1738633321,90359267,0),( 74095,1738633321,90359267,0),(74096,1738633321,903 59267,0),(75241,1738633321,148727078,4);."
932:20250204:024208.543 slow query: 4.061145 sec, "insert into history_uint (itemid,clock,ns,value) values (32043,1738633323,594740254,822468608),(32163,1738 633323,140303613,0),(32223,1738633323,594885570,0) ,(32823,1738633323,140042907,1476395008),(32883,17 38633323,148726401,248),(32943,1738633323,14045191 6,0),(33003,1738633323,140140809,0),(33063,1738633 323,141232834,255),(33183,1738633323,140904026,948 584448),(33663,1738633323,140530918,0),(33843,1738 633323,594723129,0),(33903,1738633323,595893740,0) ,(33963,1738633323,140268712,255),(34023,173863332 3,140391015,255),(34143,1738633323,594669233,1),(3 8522,1738633322,964035943,0),(39782,1738633322,742 432118,472872),(40502,1738633322,705858274,4864),( 41043,1738633323,173342689,1),(45938,1738633320,60 5538907,1),(47584,1738633322,608794929,1),(48862,1 738633322,139836560,1),(49143,1738633323,163337622 ,3767),(50763,1738633323,161874095,11176),(51101,1 738633322,753303480,1),(51940,1738633323,66466441, 0),(54513,1738633323,223172990,2876640),(55143,173 8633323,223084990,1342177280),(55923,1738633323,56 8778240,4921704463),(56213,1738633321,315819352,1) ,(56222,1738633320,971566894,1),(61387,1738633324, 69628794,1),(64443,1738633323,144675311,20176),(64 563,1738633323,598261929,181682176),(65883,1738633 323,186197643,24520),(67263,1738633323,594791890,3 04093),(70244,1738633323,153747413,0),(74763,17386 33323,173471782,1),(75243,1738633323,774675767,1); ."
899:20250204:024208.546 slow query: 10.488845 sec, "insert into history_uint (itemid,clock,ns,value) values (32157,1738633317,143290709,0),(32817,1738633317,1 45883867,76),(32937,1738633317,142864499,0),(32997 ,1738633317,140596249,0),(33057,1738633317,1409412 57,255),(33657,1738633317,140653750,0),(33957,1738 633317,140163539,255),(34017,1738633317,140054637, 255),(38877,1738633317,71154091,1),(44440,17386333 17,203392369,2),(44441,1738633317,203392369,1),(44 442,1738633317,203392369,2),(44443,1738633317,2033 92369,2),(44444,1738633317,203392369,2),(44445,173 8633317,203392369,1),(44448,1738633317,203392369,2 ),(44482,1738633317,203392369,2),(44483,1738633317 ,203392369,2),(44484,1738633317,203392369,2),(4448 5,1738633317,203392369,2),(44486,1738633317,203392 369,2),(44487,1738633317,203392369,2),(44488,17386 33317,203392369,2),(44489,1738633317,203392369,1), (44490,1738633317,203392369,1),(44491,1738633317,2 03392369,1),(44496,1738633317,203392369,2),(44497, 1738633317,203392369,2),(44510,1738633317,20339236 9,2),(44511,1738633317,203392369,2),(44512,1738633 317,203392369,2),(44513,1738633317,203392369,2),(4 4514,1738633317,203392369,2),(44515,1738633317,203 392369,2),(44516,1738633317,203392369,2),(44517,17 38633317,203392369,2),(44518,1738633317,203392369, 2),(44519,1738633317,203392369,2),(44570,173863331 7,203392369,2),(44571,1738633317,203392369,2),(445 72,1738633317,203392369,2),(44573,1738633317,20339 2369,2),(44574,1738633317,203392369,2),(44575,1738 633317,203392369,2),(44576,1738633317,203392369,2) ,(44577,1738633317,203392369,2),(44578,1738633317, 203392369,2),(44579,1738633317,203392369,2),(44580 ,1738633317,203392369,47),(44581,1738633317,203392 369,45),(44582,1738633317,203392369,38),(44583,173 8633317,203392369,36),(44584,1738633317,203392369, 47),(44585,1738633317,203392369,47),(44586,1738633 317,203392369,47),(44587,1738633317,203392369,45), (44588,1738633317,203392369,55),(44589,1738633317, 203392369,42),(44590,1738633317,203392369,2),(4459 1,1738633317,203392369,2),(44592,1738633317,203392 369,2),(44593,1738633317,203392369,2),(44594,17386 33317,203392369,2),(44595,1738633317,203392369,2), (44596,1738633317,203392369,4),(44597,1738633317,2 03392369,4),(44598,1738633317,203392369,4),(44599, 1738633317,203392369,4),(44600,1738633317,20339236 9,4),(44601,1738633317,203392369,4),(52197,1738633 317,227266363,11072),(54656,1738633317,203392369,2 ),(54662,1738633317,203392369,2),(54663,1738633317 ,203392369,42),(54777,1738633317,224012164,1),(548 37,1738633317,223705964,1086455808),(55077,1738633 317,149248442,0),(55137,1738633317,223455664,1106) ,(56517,1738633317,553063460,1),(56934,1738633317, 170946626,2),(59288,1738633317,5473203,0),(59289,1 738633317,5473203,0),(59290,1738633317,5473203,0), (59291,1738633317,5473203,0),(59292,1738633317,547 3203,0),(59293,1738633317,5473203,0),(59294,173863 3317,5473203,0),(59295,1738633317,5473203,0),(6257 3,1738633317,30753099,0),(62574,1738633317,3075309 9,0),(62575,1738633317,30753099,0),(62576,17386333 17,30753099,0),(62577,1738633317,30753099,0),(6257 8,1738633317,30753099,0),(62579,1738633317,3075309 9,0),(62580,1738633317,30753099,0),(63416,17386333 17,55995798,0),(63417,1738633317,55995798,0),(6369 8,1738633317,55995798,0),(63699,1738633317,5599579 8,0),(63700,1738633317,55995798,0),(63701,17386333 17,55995798,0),(63702,1738633317,55995798,0),(6370 3,1738633317,55995798,0),(64076,1738633316,9581908 04,3026488),(64437,1738633317,145153251,317656),(6 4659,1738633315,493803705,1),(67117,1738633317,559 95798,0),(67122,1738633317,5473203,0),(67127,17386 33317,30753099,0),(69956,1738633316,957553349,1540 46464),(70676,1738633316,957592486,1023406080),(71 997,1738633317,143363911,0),(72057,1738633317,5531 02335,1),(75237,1738633317,140692851,3863923);."
942:20250204:024208.546 slow query: 9.087276 sec, "insert into history_uint (itemid,clock,ns,value) values (31978,1738633318,141468080,171687936),(32158,1738 633318,140232553,0),(32218,1738633318,594530758,0) ,(32938,1738633318,141175174,0),(32998,1738633318, 141346278,0),(33058,1738633318,141384378,255),(336 58,1738633318,141504881,0),(33838,1738633318,59433 8545,0),(33898,1738633318,594948274,0),(33958,1738 633318,141054271,255),(34018,1738633318,141421379, 255),(34078,1738633318,141733386,255),(38037,17386 33317,957300950,1),(38338,1738633318,223791386,147 3),(48829,1738633316,495482055,1),(48904,173863331 8,5235041,3552599),(48917,1738633317,129345381,1), (49318,1738633318,178194818,2905055232),(50758,173 8633318,162796773,0),(52198,1738633318,227408085,0 ),(60708,1738633318,81809122,0),(60709,1738633318, 81809122,0),(60710,1738633318,81809122,0),(60711,1 738633318,81809122,0),(60712,1738633318,81809122,0 ),(60713,1738633318,81809122,0),(60714,1738633318, 81809122,0),(60715,1738633318,81809122,0),(64077,1 738633317,958207855,6974324736),(64618,1738633318, 598653717,0),(67107,1738633318,81809122,0),(67258, 1738633318,594146052,1738633318),(69957,1738633317 ,958156488,12286915),(70198,1738633318,141862489,1 625406),(71998,1738633318,139934446,0),(74434,1738 633318,22932922,0),(74435,1738633318,22932922,0),( 74436,1738633318,22932922,0),(74437,1738633318,229 32922,0),(74438,1738633318,22932922,0),(74439,1738 633318,22932922,0),(74440,1738633318,22932922,0),( 74441,1738633318,22932922,0),(74442,1738633318,229 32922,0),(75238,1738633318,144370345,17178873856), (75701,1738633318,626830630,10000000000);."
920:20250204:024208.558 slow query: 7.090167 sec, "insert into history_uint (itemid,clock,ns,value) values (32040,1738633320,593950009,12292345),(32160,17386 33320,139277455,0),(32220,1738633320,594326729,0), (32820,1738633320,139674764,1738633320),(32940,173 8633320,141078795,0),(33000,1738633320,141308101,6 ),(33060,1738633320,139842168,255),(33660,17386333 20,140066673,255),(33840,1738633320,593961187,0),( 33900,1738633320,596001463,0),(33960,1738633320,13 9638063,255),(34020,1738633320,139715665,255),(397 79,1738633319,732259107,0),(40200,1738633320,56000 4124,34226859982),(40319,1738633319,703950086,1182 96576),(40379,1738633319,702352662,207728640),(446 21,1738633318,496709406,1),(48849,1738633320,21021 117,40458885),(48860,1738633321,32341494,40458910) ,(48895,1738633318,496709406,1),(50760,1738633320, 162952812,11136),(51899,1738633319,994164369,21610 12736),(53523,1738633321,70415226,648987),(54840,1 738633320,249568724,32),(55140,1738633320,22332342 8,1738633319),(55979,1738633319,704349692,47939111 7312),(56132,1738633318,336200480,1),(56640,173863 3320,140628485,12293349),(64080,1738633320,5562051 52,1),(64748,1738633318,483503460,1),(66822,173863 3320,78909649,0),(66823,1738633320,78909649,0),(66 824,1738633320,78909649,0),(66825,1738633320,78909 649,0),(66826,1738633320,78909649,0),(66827,173863 3320,78909649,0),(66828,1738633320,78909649,0),(66 829,1738633320,78909649,0),(67097,1738633320,78909 649,0),(67260,1738633320,560090917,8589934592),(70 200,1738633320,141362602,1841545216),(72000,173863 3320,140030972,1),(72056,1738633320,93785402,54812 75),(72277,1738633320,93785402,0),(72280,173863332 0,93785402,0),(72282,1738633320,93785402,152),(722 84,1738633320,93785402,0),(72285,1738633320,937854 02,4904),(72287,1738633320,93785402,0),(72288,1738 633320,93785402,0),(72289,1738633320,93785402,0),( 72325,1738633320,93785402,16280),(72326,1738633320 ,93785402,9040),(72327,1738633320,93785402,0),(723 29,1738633320,93785402,0),(72330,1738633320,937854 02,584),(72332,1738633320,93785402,0),(72334,17386 33320,93785402,0),(72337,1738633320,93785402,7368) ,(72341,1738633320,93785402,0),(72424,1738633320,9 3785402,0),(72425,1738633320,93785402,0),(72426,17 38633320,93785402,0),(72427,1738633320,93785402,0) ,(72428,1738633320,93785402,0),(72429,1738633320,9 3785402,0),(72430,1738633320,93785402,0),(72431,17 38633320,93785402,0),(72432,1738633320,93785402,0) ,(74487,1738633320,32727962,0),(74488,1738633320,3 2727962,0),(74489,1738633320,32727962,0),(74490,17 38633320,32727962,0),(74491,1738633320,32727962,0) ,(74492,1738633320,32727962,0),(74493,1738633320,3 2727962,0),(74494,1738633320,32727962,0),(74495,17 38633320,32727962,0);."
922:20250204:024208.558 slow query: 8.090851 sec, "insert into history_uint (itemid,clock,ns,value) values (32039,1738633319,593030604,610275328),(32159,1738 633319,141478393,0),(32219,1738633319,593106098,0) ,(32939,1738633319,141528894,0),(32999,1738633319, 142149307,0),(33059,1738633319,141624296,255),(331 79,1738633319,141888902,1738633318),(33509,1738633 319,592683214,2951202),(33659,1738633319,142185008 ,255),(33839,1738633319,593012601,0),(33899,173863 3319,594241645,0),(33959,1738633319,141660397,255) ,(34019,1738633319,141752199,255),(34079,173863331 9,141796000,255),(38039,1738633319,6752248,1521008 64),(38339,1738633319,228863806,91),(39778,1738633 318,738710617,0),(40198,1738633318,700589849,17386 33318),(40438,1738633318,700640750,1),(46138,17386 33318,701261759,189333266432),(46239,1738633317,66 1156682,1),(46242,1738633316,599093466,1),(46904,1 738633318,136818936,1),(48839,1738633319,770289322 ,1),(50126,1738633320,54357828,3),(50759,173863331 9,160357495,0),(51898,1738633318,994573096,4293943 296),(52199,1738633319,228906606,0),(54440,1738633 319,59987106,100),(54441,1738633319,59987106,2),(5 4446,1738633319,59987106,2),(54911,1738633319,6494 13731,0),(55947,1738633316,599093466,1),(56077,173 8633317,367050898,1),(56398,1738633318,701243059,1 45469440),(56639,1738633319,552805012,8474230750), (57518,1738633316,969438809,1),(57521,1738633317,1 54709836,1),(57631,1738633319,649413731,0),(64559, 1738633319,593577567,16221221),(64745,1738633317,2 80847023,1),(70019,1738633319,224245207,88059904), (70199,1738633319,142765121,17178873856),(70678,17 38633318,958137277,143110),(70679,1738633319,12720 948,0),(71641,1738633318,810186599,0),(71649,17386 33318,809743766,200),(75239,1738633319,141158985,3 395416064),(75659,1738633319,609505188,1);."
944:20250204:024208.564 slow query: 3.074726 sec, "insert into history_uint (itemid,clock,ns,value) values (31405,1738633322,502805189,1),(32164,1738633324,1 41203844,0),(32224,1738633324,593565662,0),(32944, 1738633324,141912960,0),(33004,1738633324,14160955 4,0),(33064,1738633324,141569253,255),(33574,17386 33324,594214937,2957575),(33664,1738633324,1420311 63,255),(33844,1738633324,593222434,0),(33904,1738 633324,594599952,0),(33964,1738633324,141830458,25 5),(34024,1738633324,141681155,255),(37341,1738633 322,502805189,1),(38043,1738633323,957969306,17840 1280),(38134,1738633324,223343411,4091300),(38344, 1738633324,576084789,2422321143),(38523,1738633323 ,964142057,7152),(40203,1738633323,701396394,27842 31),(40323,1738633323,701806100,136843264),(40503, 1738633323,706872776,0),(45003,1738633323,70811019 4,0),(45904,1738633324,264027170,0),(49144,1738633 324,171545043,213),(50764,1738633324,160687141,1), (51903,1738633323,630968976,1),(53524,1738633324,6 14563372,1),(55144,1738633324,223481911,2884933),( 56216,1738633322,697413701,1),(56644,1738633324,14 2389971,122945536),(63184,1738633324,78550637,1),( 63664,1738633324,78618737,1),(64444,1738633324,144 467817,0),(64757,1738633321,697104200,1),(64760,17 38633322,697413701,1),(65674,1738633324,696508434, 255),(65675,1738633324,696508434,255),(65692,17386 33324,696508434,27),(65693,1738633324,696508434,27 ),(65694,1738633324,696508434,27),(65695,173863332 4,696082125,27),(65696,1738633324,696508434,27),(6 5697,1738633324,696508434,27),(65698,1738633324,69 6508434,28),(65699,1738633324,696508434,28),(65700 ,1738633324,696508434,28),(65701,1738633324,696508 434,28),(65702,1738633324,696508434,28),(65703,173 8633324,696508434,29),(65746,1738633324,696508434, 18),(65747,1738633324,696508434,28),(65748,1738633 324,696508434,39),(65750,1738633324,696508434,55), (65752,1738633324,696508434,28),(65753,1738633324, 696508434,30),(65754,1738633324,696508434,29),(657 55,1738633324,696508434,30),(65756,1738633324,6965 08434,32),(65757,1738633324,696508434,34),(65758,1 738633324,696508434,33),(65759,1738633324,69650843 4,32),(65760,1738633324,696508434,34),(65884,17386 33324,182085459,0),(67144,1738633324,600938332,0), (67264,1738633324,593280493,4293971968),(70204,173 8633324,775938206,1),(70684,1738633324,560322688,1 ),(71044,1738633324,107845254,1),(71284,1738633324 ,97748657,0),(71464,1738633324,88151559,1),(71524, 1738633324,16891916,0),(74824,1738633324,576054036 ,0);."
941:20250204:024248.687 item "HOST: power.cons.others" became not supported: Value of type "string" is not suitable for value type "Numeric (unsigned)". Value "-65371"
899:20250204:024347.963 item "HOST: power.cons.others" became supported
891:20250204:024455.659 executing housekeeper
891:20250204:024515.003 housekeeper [deleted 228129 hist/trends, 0 items/triggers, 4 events, 4 problems, 0 sessions, 0 alarms, 435 audit, 0 autoreg_host, 0 records in 19.344144 sec, idle for 1 hour(s)]
1677:20250204:025937.582 Proxy "PROXY" changed state from online to offline
891:20250204:034515.089 executing housekeeper
818:20250204:064911.281 Starting Zabbix Server. Zabbix 7.0.9 (revision 05b8b05eefe).

The last line is after I reseted the server and Zabbix came back to life.

I don't know if such SQL queries are OK.

Hours before the crash, I have a lot of "became not supported: Cannot obtain performance information from collector." from 1-2 hosts who never cause problem before.
I have such behavor yesterday with other hosts I disabled for tests purpose, but seems other hosts take relay now...

Thanks for help
N.

**MRedbourne** · 04-02-2025, 08:25

The SQL alarms aren't particularly good... For context, I've never thrown a slow DB log before, and my DB is relatively small. 2vCore and 4GB of RAM IIRC. NVPS around 500. What type of load is this server under? The error logs indicate your DB is being hosted locally.

> Aborted connection 103 to db: 'zabbix' user: 'zabbix' host: 'localhost' (Got timeout reading communication packets)

Might be time to move the DB off the server and onto a dedicated host, or a hosted service for you. Or at the very least, if the zabbix server is being overrun with tasks, increasing it's CPU and memory.

**nicolasfo** · 04-02-2025, 08:52

The Zabbix server don't seems to be overbusy, but I increased the vCPU from 4 to 8, the RAM was at 32G and I don't touch it.
I migrate the VM VD on full NVME storage, let's see what happen...

Before crash, I didn't had host for 1-2 months, so the load didn't increase from day to day...
Nothing changed for this server, except the update from 7.0.7 to 7.0.9 if my memory is correct.

Thanks

**colizab** · 04-02-2025, 09:41

hey,
I've had about the same problem for 2 weeks now
the server crashes alone without explanation and I get the same message in the database logs
the zabbix servers restart with sudo systemctl restart zabbix-server but not the database, which also forces me to restart from the vmware console
If you have a solution, I'm interested !

**mateobrl** · 04-02-2025, 10:28

Hello, I've had the same problem as you for 2 weeks. Knowing that I redid my entire zabbix once with my old database and another time from 0 complete. And I'm having the same problem over and over again.

**nicolasfo** · 04-02-2025, 12:24

Originally posted by colizab

hey,
I've had about the same problem for 2 weeks now
the server crashes alone without explanation and I get the same message in the database logs
the zabbix servers restart with sudo systemctl restart zabbix-server but not the database, which also forces me to restart from the vmware console
If you have a solution, I'm interested !

You don't resolve my problem, but I'm happy I'm not alone... Thanks !

Did you update to 7.0.9 ?

Thanks
N

**nicolasfo** · 04-02-2025, 19:15

Originally posted by mateobrl

Hello, I've had the same problem as you for 2 weeks. Knowing that I redid my entire zabbix once with my old database and another time from 0 complete. And I'm having the same problem over and over again.

What's the version of your Zabbix server ?

N

**mateobrl** · 05-02-2025, 09:43

Originally posted by nicolasfo

What's the version of your Zabbix server ?

N

My zabbix is in 7.0.9. But initially it was in 7.2.0 and I had the same problem.

**BenjaminG** · 06-02-2025, 23:39

Hi

I have the same issue since last week. Same Mariadb logs and "normal" zabbix_server logs.
Upgraded 7.0.4 to 7.0.9.
Mariadb 10.11 initially, upgraded to 11.4.6 in the meantime
Debian 12

Its a rather old installation that went from 5.x to 6.0 to 7.0.

I find no useful logs whatsoever. mariadb (or better the zabbix db) basically just locks up every few hours. Sometimes after 1h, sometimes over 24h.
When it happens, it feels like a deadlock and there are locked entities in the processlist, but I cannot find any lock source query.
I can not stop the mariadb service nor zabbix server. I can only sigkill mariadb or reset the server.

I am completely lost with this case

**colizab** · 07-02-2025, 10:07

hello, I may have a lead
i had an almost daily crash for 2 weeks and it's been 1 day since i had one
I saw this log in my mariadb logs when crash:
“srv-zabbix mariadbd [Warning] InnoDB: Could not free any blocks in the buffer pool! 8064 blocks are in use and 0 free. Consider increasing innodb_buffer_pool_size.”

I changed the innobdb value:

/etc/mysql/mariadb.conf.d/50-server.cnf
innodb_buffer_pool_size = 4G

set it to 4G

then I had a problem when restarting the mariadb service, which wouldn't start because provider bzip2/lz4 couldn't find it ... I commented out each line for each item that was causing problems in /etc/mysql/mariadb.conf.d

it's been more than a day and there's no blockage so far, keep an eye on it after the weekend

**hookproto** · 07-02-2025, 10:14

We've got multiple Zabbix instances for different purposes. On one setup the main server runs on Debian 11 and has been rock solid. It has several proxies connected to it, but after upgrading one of the proxies to Debian 12, we started seeing random crashes on that proxy. You can’t even stop the service with systemctl; you have to manually kill all Zabbix processes.

In another setup, we upgraded Debian to 12 on a machine running the Zabbix server itself, and it’s showing the exact same issue as the proxy.

Edit:
During the upgrade to Debian 12, the Zabbix proxy was also updated from version 6.4.20 to 6.4.21, which might be relevant

On one of the Zabbix servers, the zabbix version was upgraded from 6.4.20 to 7.0.3 and then to 7.2.3, along with the Debian 12 upgrade.

**BenjaminG** · 07-02-2025, 11:26

Originally posted by colizab

hello, I may have a lead
i had an almost daily crash for 2 weeks and it's been 1 day since i had one
I saw this log in my mariadb logs when crash:
“srv-zabbix mariadbd [Warning] InnoDB: Could not free any blocks in the buffer pool! 8064 blocks are in use and 0 free. Consider increasing innodb_buffer_pool_size.”

I changed the innobdb value:

/etc/mysql/mariadb.conf.d/50-server.cnf
innodb_buffer_pool_size = 4G

set it to 4G

then I had a problem when restarting the mariadb service, which wouldn't start because provider bzip2/lz4 couldn't find it ... I commented out each line for each item that was causing problems in /etc/mysql/mariadb.conf.d

it's been more than a day and there's no blockage so far, keep an eye on it after the weekend

I also had this exact issue happen the first time with the begining of these problems.
I found that my innodb_buffer_pool_size wasnt set and therefore defaulted to 128MB. I set that to 2G and havent seen this message since. Crashes happen anyways

**nicolasfo** · 07-02-2025, 11:41

Hello,
According to Colizab, I changed innodb_buffer_pool_size to 8G (I have 32G RAM on this server), without any problem when restarting.

Hookproto, what's the point ? It's over my knowledge...

Thanks
N

Ad Widget

Zabbix server crashes

Zabbix server crashes

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment