Ad Widget

Collapse

Zabbix History Syncer suddenly spikes and keeps high on IO Usage [Repost]

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • DrNick
    Junior Member
    • Aug 2026
    • 11

    #1

    Zabbix History Syncer suddenly spikes and keeps high on IO Usage [Repost]

    ***sorry for the repost, lost connection due to cloudflare, and seemed that i also lost a big part of my text and couldnt edit post***

    Hey there, hope that someone can help me with some pointers where to look at, i don't have all the log files of the event, so i cant bee that specific.

    We have a small zabbix server running for quite some time, its hosted on a Ubuntu 24.04.2 LTS; MySQL ; Zabbix version 7.0.7, Running as VM on a Windows Hyper-V.
    About 140 hosts monitored; about 10.500 active items; with a nvps of about 50.

    The server has been running fine, no problems at all, low io, nice usage, nothing to be concerend about.
    But when i was on holiday, suddenly on Sunday the 19th of July roughly between 17:00 and 19:00 our server IO Spiked and basically never returned to normal again.
    IO Wait times of 50% or more and that without stopping.

    Stopping the zabbix server, didn't lower it instantly; eventually it would drop. Shutting down MySQL; would stop the IO load.
    Starting the MySQL server didn't start the IO spikes back again and would run normally.
    After starting the zabbix server, the IO spike did return after some time. As far as i could see, the History syncer was busy more then other processes.

    As far as i could see the log files from zabbix dont throw any real big issues or errors, not on startup or anything like that.
    i did enable the slow log query and that does show some more queries, but i can't really tell if that causes it, or is a result of.

    To the best of our knowledge in the recent past before the event, no new hosts have been added, no old hosts have been removed,
    and no real changes where made. Even the host system was let alone, i mostly do those things, but not this time.

    Does anyone have some clue's where to look at?
    I don't think 'performance' is in place here, because of the sudden event and the load before was not that high in any way.
    On the hardware host also everything seems to be normal, no broken disks, no array faults, only high IO usage since that event.
    (the machine is hosting 1 other VM, but no that VM isn't causing the IO load on the main host).

    Is there some option i could turn on to show more why or what is happening. In General we dont keep that much of a history, to make the server not too overloaded.
    single items stored roughly 3days to max 7days (mostly depending on poll interval); trends stored mostly max 180days;

    i added some screenshots, roughly 24hour graps, Prior to the event, during the event, after the event (with more detail) and after the mysql (+reboot) change.
    Hope that someone can point me in some direction where to look at.

    DB backup's are done 4 times a week, zabbix server is shut down, DB backup is taken, mysql is restarted, zabbix server is start up again.
    This is also running for some years and was not running during this event, that would run at night time.

    Code:
    [mysqld]
    user = mysql
    bind-address = 127.0.0.1
    mysqlx-bind-address = 127.0.0.1
    key_buffer_size = 16M
    myisam-recover-options = BACKUP
    innodb_buffer_pool_size=4G
    innodb_flush_log_at_trx_commit=2
    log_error = /var/log/mysql/error.log
    slow_query_log = 1
    slow_query_log_file = /var/log/mysql/mysql-slow.log
    long_query_time = 2
    max_binlog_size = 100M
    binlog_expire_logs_seconds = 604800
    Code:
    ListenPort=10051
    LogType=file
    LogFile=/var/log/zabbix/zabbix_server.log
    LogFileSize=5
    DebugLevel=3
    PidFile=/tmp/zabbix_server.pid
    DBHost=localhost
    DBName=zabbix
    DBUser=zabbix
    DBPassword=supersecret
    DBPort=3306
    StartPollers=10
    StartIPMIPollers=0
    StartPreprocessors=6
    StartPollersUnreachable=10
    StartTrappers=6
    StartPingers=30
    StartDiscoverers=1
    StartHTTPPollers=2
    StartTimers=2
    StartEscalators=2
    StartAlerters=2
    StartJavaPollers=0
    StartVMwareCollectors=0
    SNMPTrapperFile=/opt/zabbix/tmp/zabbix_traps.tmp
    StartSNMPTrapper=1
    ListenIP=0.0.0.0
    HousekeepingFrequency=1
    MaxHousekeeperDelete=5000
    CacheSize=64M
    StartDBSyncers=2
    HistoryCacheSize=512M
    HistoryIndexCacheSize=64M
    Timeout=30
    TrapperTimeout=300
    UnreachablePeriod=35
    UnavailableDelay=60
    UnreachableDelay=15
    ExternalScripts=/opt/zabbix/share/zabbix/externalscripts
    FpingLocation=/usr/sbin/fping
    Fping6Location=/usr/sbin/fping6
    LogSlowQueries=3000
    TmpDir=/tmp
    StartProxyPollers=1
    ProxyConfigFrequency=3600
    ProxyDataFrequency=60
    AllowRoot=0
    User=zabbix-srv
  • cyber
    Senior Member
    Zabbix Certified SpecialistZabbix Certified Professional
    • Dec 2006
    • 4957

    #2
    Based on your description, I would look for issues in DB logs.

    Comment

    • DrNick
      Junior Member
      • Aug 2026
      • 11

      #3
      The problem is that i cant see what has been changed, that maybe would have caused this behaviour.
      Yesterday i shutdown the vm, did an full copy of the disk file, to write all new sectors to the raid array, started from that new copy, no success.

      Today i made a little log file of things happening; and some things i cant explain.
      When i shutdown zabbix-server; for about 2 to 3 minutes that there are still changes made in the .idb files of the zabbix database.

      Zabbix slow query setting reduced from 3000 to 2000, not a single hit in the log file.
      Only 2 hits in the mysql slow query, but both at the time i ran a compile action. So not related.
      In the trends and trends_uint table's; i removed some old records there where not cleaned properly;
      but this where only about 5000ish records in total.

      running in mysql the 'show processlist'; does show new queries basically every time i execute the command. even when done quite fast after each other.
      Only this line keeps standing in there
      Code:
      |   20 | zabbix          | localhost | zabbix | Query   |    0 | waiting for handler commit | commit
      and this line changes every time; where i think a housekeeper was running at that time
      Code:
      |  876 | zabbix          | localhost | zabbix | Query   |    0 | waiting for handler commit | delete from history_uint where itemid=100100000059352 and clock<1785405587
      Here is some more findings.
      Code:
      # shutdown zabbix service
      root@ZabbixServer01:/etc/apache2# date; systemctl stop zabbix-server.service; date;
      Thu Aug 6 10:29:18 AM CEST 2026
      Thu Aug 6 10:29:28 AM CEST 2026
      
      # Disabled the Zabbix Site in Apache
      root@ZabbixServer01:/etc/apache2# date; systemctl reload apache2.service; date
      Thu Aug 6 10:29:45 AM CEST 2026
      Thu Aug 6 10:29:45 AM CEST 2026
      
      # Keep a track on changes
      root@ZabbixServer01:/etc/apache2# cd /var/lib/mysql/zabbix/
      root@ZabbixServer01:/var/lib/mysql/zabbix# watch "ls -lt | grep ibd$ | head -20"
      
      # End result date time
      root@ZabbixServer01:/var/lib/mysql/zabbix# ls -alht
      total 5.3G
      -rw-r----- 1 mysql mysql 1.3G Aug 6 10:32 trends_uint.ibd
      -rw-r----- 1 mysql mysql 600M Aug 6 10:32 trends.ibd
      -rw-r----- 1 mysql mysql 29M Aug 6 10:32 auditlog.ibd
      -rw-r----- 1 mysql mysql 1.3G Aug 6 10:32 history.ibd
      -rw-r----- 1 mysql mysql 1.8G Aug 6 10:32 history_uint.ibd
      -rw-r----- 1 mysql mysql 132M Aug 6 10:32 history_str.ibd
      -rw-r----- 1 mysql mysql 32M Aug 6 10:32 history_text.ibd
      -rw-r----- 1 mysql mysql 10M Aug 6 10:32 item_discovery.ibd
      -rw-r----- 1 mysql mysql 368K Aug 6 10:32 trigger_discovery.ibd
      -rw-r----- 1 mysql mysql 31M Aug 6 10:32 events.ibd
      -rw-r----- 1 mysql mysql 12M Aug 6 10:32 event_tag.ibd
      -rw-r----- 1 mysql mysql 14M Aug 6 10:32 event_recovery.ibd
      -rw-r----- 1 mysql mysql 720K Aug 6 10:32 problem.ibd
      -rw-r----- 1 mysql mysql 208K Aug 6 10:32 problem_tag.ibd
      -rw-r----- 1 mysql mysql 288K Aug 6 10:32 graph_discovery.ibd
      -rw-r----- 1 mysql mysql 160K Aug 6 10:31 escalations.ibd
      -rw-r----- 1 mysql mysql 9.0M Aug 6 10:31 item_rtdata.ibd
      -rw-r----- 1 mysql mysql 128K Aug 6 10:31 sessions.ibd
      -rw-r----- 1 mysql mysql 11M Aug 6 10:31 triggers.ibd
      -rw-r----- 1 mysql mysql 176K Aug 6 10:31 interface.ibd
      -rw-r----- 1 mysql mysql 144K Aug 6 10:31 ha_node.ibd
      -rw-r----- 1 mysql mysql 624K Aug 6 10:30 profiles.ibd
      -rw-r----- 1 mysql mysql 192K Aug 6 10:30 event_suppress.ibd
      -rw-r----- 1 mysql mysql 12M Aug 6 10:30 changelog.ibd
      drwx------ 10 mysql mysql 4.0K Aug 6 09:23 ..
      -rw-r----- 1 mysql mysql 224K Aug 6 08:26 config.ibd
      -rw-r----- 1 mysql mysql 15M Aug 6 05:43 alerts.ibd
      
      
      # Shutting down mysql
      root@ZabbixServer01:/opt/zabbix/etc# date; systemctl stop mysql.service; date
      Thu Aug 6 11:14:05 AM CEST 2026
      Thu Aug 6 11:14:08 AM CEST 2026
      
      # Starting MySQL back up
      root@ZabbixServer01:/home/User01# date; systemctl start mysql.service ; date
      Thu Aug 6 11:25:12 AM CEST 2026
      Thu Aug 6 11:25:19 AM CEST 2026
      
      # mysql error log on startup (waited 3 mins)
      ==> error.log <==
      2026-08-06T09:25:13.125926Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.46-0ubuntu0.24.04.3) starting as process 254093
      2026-08-06T09:25:13.175496Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
      2026-08-06T09:25:16.775757Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
      2026-08-06T09:25:19.025303Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
      2026-08-06T09:25:19.025360Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
      2026-08-06T09:25:19.194343Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '127.0.0.1' port: 33060, socket: /var/run/mysqld/mysqlx.sock
      2026-08-06T09:25:19.194443Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.46-0ubuntu0.24.04.3' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu).
      
      # Starting zabbix server back up
      root@ZabbixServer01:/home/User01# date; systemctl start zabbix-server.service; date
      Thu Aug 6 11:29:46 AM CEST 2026
      Thu Aug 6 11:29:46 AM CEST 2026
      
      ## 1 hit in mysql error log
      2026-08-06T09:29:46.515061Z 8 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ''mysql_native_password' is deprecated and will be removed in a future release. Please use caching_sha2_password instead'
      
      
      root@ZabbixServer01:/var/log/zabbix# tail -f zabbix_server.log
      254296:20260806:112946.505 Starting Zabbix Server. Zabbix 7.0.7 (revision af5010ccdbb).
      254296:20260806:112946.506 ****** Enabled features ******
      254296:20260806:112946.506 SNMP monitoring: YES
      254296:20260806:112946.506 IPMI monitoring: YES
      254296:20260806:112946.506 Web monitoring: YES
      254296:20260806:112946.506 VMware monitoring: YES
      254296:20260806:112946.506 SMTP authentication: YES
      254296:20260806:112946.506 ODBC: NO
      254296:20260806:112946.506 SSH support: YES
      254296:20260806:112946.506 IPv6 support: YES
      254296:20260806:112946.506 TLS support: YES
      254296:20260806:112946.506 ******************************
      254296:20260806:112946.506 using configuration file: /opt/zabbix/etc/zabbix_server.conf
      254296:20260806:112946.689 current database version (mandatory/optional): 07000000/07000015
      254296:20260806:112946.689 required mandatory version: 07000000
      254296:20260806:112946.694 database could be upgraded to use primary keys in history tables
      254300:20260806:112946.709 starting HA manager
      254300:20260806:112946.769 HA manager started in active mode
      (...)
      254359:20260806:112952.551 forced reloading of the snmp cache on [unreachable poller #10]
      
      254413:20260806:113209.602 Zabbix agent item "xx" on host "server1" failed: first network error, wait for 15 seconds
      254413:20260806:113224.604 resuming Zabbix agent checks on host "server1": connection restored
      254324:20260806:113228.810 item "ZabbixServer01:zabbix[process,proxy poller,avg,busy]" became not supported: No "proxy poller" processes started.
      254413:20260806:113411.603 Zabbix agent item "xx" on host "server1" failed: first network error, wait for 15 seconds
      (...)
      
      
      # Restarting web interface
      root@ZabbixServer01:/home/User01# a2ensite zabbix-ssl.conf
      Enabling site zabbix-ssl.
      To activate the new configuration, you need to run:
      systemctl reload apache2
      root@ZabbixServer01:/home/User01# date; systemctl reload apache2.service ; date
      Thu Aug 6 11:43:19 AM CEST 2026
      Thu Aug 6 11:43:19 AM CEST 2026
      
      
      ### 11:50:00
      ### Nore more new other logs (exept networkerror-restored) in both mysql,zabbix,mysql-slow log files.
      Little overview table usage from phpmyadmin:
      Click image for larger version

Name:	phpmyadmin.jpg
Views:	0
Size:	53.0 KB
ID:	515722

      Little screenshot, load on host after Starting zabbix server:
      Click image for larger version

Name:	Start-Zabbix-Server.jpg
Views:	0
Size:	120.8 KB
ID:	515721
      Attached Files

      Comment

      Working...