Ad Widget

Collapse

zabbix server history_uint long running query

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • yunhyeonglee
    Junior Member
    • Jul 2024
    • 4

    #1

    zabbix server history_uint long running query

    Currently, the environment is composed of Zabbix Proxy (6 instances) and a Zabbix Server (1 instance).
    The history, history_uint, and trends tables are partitioned by clock on a daily basis.

    The minimum clock value present in those tables is 1717167600.

    However, Zabbix Server internally executes queries like the following against history / history_uint, which causes long-running queries due to overly broad time ranges:

    SELECT clock, ns, value FROM history_uint WHERE itemid = 663187 AND clock > 1689304026 AND clock <= 1749458997;

    It is confirmed that this query is not triggered when viewing graphs via the Zabbix frontend, and the query is executed under the user defined in the zabbix_server.conf file.
    Why does Zabbix internally issue such queries with unnecessarily wide time ranges, causing long-running queries, even though the minimum data starts from 1717167600?

    The issue persists even after restarting the Zabbix server.
    These long-running queries are causing service issues within Zabbix.
  • Markku
    Senior Member
    Zabbix Certified SpecialistZabbix Certified ProfessionalZabbix Certified Expert
    • Sep 2018
    • 1781

    #2
    What is your Zabbix version?

    Markku

    Comment

    • yunhyeonglee
      Junior Member
      • Jul 2024
      • 4

      #3
      My Questions Regarding Zabbix Behavior After Restart
      Environment
      OS: Rocky Linux release 8.6
      Zabbix Version: zabbix_server (Zabbix) 6.4.1, compiled on April 10, 2023


      1. Why does Zabbix scan the history and history_uint tables after the server restarts?
      Why is Zabbix internally scanning historical data (specifically history / history_uint) for individual itemids immediately after the service restarts?
      What internal purpose does this serve?

      SELECT clock, ns, value FROM history WHERE itemid = 745785 AND clock > 1702790748 AND clock <= 1749510445;
      SELECT clock, ns, value FROM history_uint WHERE itemid = 663187 AND clock > 1689304026 AND clock <= 1749458997;

      2.How long do these post-restart history scans continue?
      We’ve observed that even after the scan for one itemid completes, Zabbix initiates new scan queries for other itemids.
      When (or under what conditions) does Zabbix stop this behavior?
      Is there a threshold, completion condition, or configuration parameter that defines when these scans end?


      3.During this scan period, metrics are collected and triggers are evaluated correctly — but actions do not execute. Why?
      We’ve confirmed that data collection and trigger evaluation are working normally during this time.
      However, actions (such as notifications) are not operated until after the history scans are completed.

      Comment

      • yunhyeonglee
        Junior Member
        • Jul 2024
        • 4

        #4
        If querying history table data is internally required after Zabbix is restarted, is there a way for those queries to run in parallel rather than being processed sequentially in a single thread?

        Currently, the queries are executed like this:
        SELECT clock, ns, value FROM history_uint
        WHERE itemid = 857412
        AND clock > 1710322102
        AND clock <= 1749522346;
        Once the above query completes, then the next one is executed:

        SELECT clock, ns, value FROM history_uint
        WHERE itemid = 857440
        AND clock > 1710322102
        AND clock <= 1749522346;


        Is there any method that allows these queries—or even more—to be executed simultaneously, rather than one after another?
        For example, executing both of the following at the same time:

        SELECT clock, ns, value FROM history_uint
        WHERE itemid = 857412
        AND clock > 1710322102
        AND clock <= 1749522346;

        SELECT clock, ns, value FROM history_uint
        WHERE itemid = 857440
        AND clock > 1710322102
        AND clock <= 1749522346;

        Is there any internal method or configuration that would allow such queries to run in parallel, not sequentially?

        Comment

        • PavelZ
          Senior Member
          • Dec 2024
          • 162

          #5
          Queries are executed sequentially because the scheduler tries to spread the load evenly over time.
          Queries occur after a restart because this information is not in the value cache in RAM.

          It makes no sense to consider item ID separately from what the server does. Tell us more about triggers and calculated items.

          Comment

          • PavelZ
            Senior Member
            • Dec 2024
            • 162

            #6
            The history, history_uint, and trends tables are partitioned by clock on a daily basis.
            This is all good, but have you disabled housekeeping in Zabbix settings?

            Comment

            • yunhyeonglee
              Junior Member
              • Jul 2024
              • 4

              #7
              The issue has been resolved.
              In my case, as of the current date (2025/06/11), I was receiving alerts for unresolved problems that occurred as early as 2023-12-17 14:25:48 (clock:1702790748).
              At that point, the query below was executed:
              SELECT clock, ns, value FROM history WHERE itemid = 745785 AND clock > 1702790748 AND clock <= 1749510445;
              As soon as the query finished running, I received an alert for that item.

              [Unresolved Problems]
              2023: 20 entries
              2024: 40 entries
              So what I did was delete the data for old events, and once those outdated events were removed, I started receiving alerts for newly occurring events. >> DELETE FROM event WHERE clock > 1702790748 AND clock < 1749410445 AND value=1 AND source=0;

              However, I have a question:
              When Zabbix or its database is restarted,
              does Zabbix internally send alerts for unresolved problems?
              Is there an internal execution sequence that runs when Zabbix and its DB are restarted?
              If such a process exists, I would appreciate it if you could share any related documentation.
              Let me know if you'd like help looking into official documentation or community references about Zabbix’s behavior during service restarts.

              Comment

              Working...