Ad Widget

Collapse

Zabbix database overloaded by UPDATE statements

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • lguillau
    Junior Member
    • Jul 2022
    • 7

    #1

    Zabbix database overloaded by UPDATE statements

    Hello guys,

    I'm running Zabbix 5.0.24 with 30 proxies, a PostgreSQL + TimescaleDB backend with a 2 To database (14000 hosts, 1,1 millions of items).

    When I perform maintenance tasks on Zabbix which require to restart the zabbix-server process (minor upgrade for example), the database is overloaded by statements like "UPDATE trends_uint SET...". As the database is fully busy by these requests, no new data are inserted to the history tables during at least an hour and it creates a lot of chaos in my monitoring system.

    The documentation says:

    When server flushes trend cache and there are already trends in the database for this hour (for example, server has been restarted mid-hour), server needs to use update statements instead of simple inserts. Therefore on a bigger installation if restart is needed it is desirable to stop server in the end of one hour and start in the beginning of the next hour to avoid trend data overlap.
    So if I understand correctly, should I stop Zabbix at 6:58 for example, and start it at 7:02 in order to avoid overlap? The last time I did this, I fell again in the "UPDATE" situation (screenshot below).
    Zabbix server queue issue after restart



    Am I missing something? What's the good process to restart Zabbix without this kind of issue please?

    Thank you!
  • LenR
    Senior Member
    • Sep 2009
    • 1005

    #2
    I suspect it's the data buffered by your proxies during the outage. What is your NVPS value? Can you tune your DB for faster write? Check the DB performance stats during that time. Are you using SSD disk?

    Comment

    • cyber
      Senior Member
      Zabbix Certified SpecialistZabbix Certified Professional
      • Dec 2006
      • 4806

      #3
      Try to stop it at 6:54.

      Server flushes trends into the database
      • a new hour has started and server receives a new value for the item;
      • a new hour is about to end in less than 5 minutes (no new values)
      • server stops

      Comment

      • lguillau
        Junior Member
        • Jul 2022
        • 7

        #4
        Originally posted by LenR
        I suspect it's the data buffered by your proxies during the outage. What is your NVPS value? Can you tune your DB for faster write? Check the DB performance stats during that time. Are you using SSD disk?
        Indeed, I have to stop the proxies too, before stopping zabbix-server. Thank you for your help.

        Comment

        Working...