Ad Widget

Collapse

fuzzy time generates false positives

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • alj
    Senior Member
    • Aug 2006
    • 188

    #1

    fuzzy time generates false positives

    it seems like system.localtime metric gets delayed somewhere within server when server/database is too busy and fuzzy time trigger which looks like this:
    {template:system.localtime.fuzzytime(5)}#1
    generates lots of false positives.

    Is there any way to avoid that?
  • Alexei
    Founder, CEO
    Zabbix Certified Trainer
    Zabbix Certified SpecialistZabbix Certified Professional
    • Sep 2004
    • 5654

    #2
    Hmm, interesting question. Even if the processing will be super-fast, we still have a chance of slow comunications, slow-downs on agent side, etc.

    I think that the best solution would be expanding of the function to support second parameter which would specify how many last values we compare with ZABBIX server's system time. Like:

    fuzzytime(5, 3) - if all 3 most recent values are within 5 seconds range, return 0

    Will it help?
    Alexei Vladishev
    Creator of Zabbix, Product manager
    New York | Tokyo | Riga
    My Twitter

    Comment

    • alj
      Senior Member
      • Aug 2006
      • 188

      #3
      Originally posted by Alexei
      Hmm, interesting question. Even if the processing will be super-fast, we still have a chance of slow comunications, slow-downs on agent side, etc.

      I think that the best solution would be expanding of the function to support second parameter which would specify how many last values we compare with ZABBIX server's system time. Like:

      fuzzytime(5, 3) - if all 3 most recent values are within 5 seconds range, return 0

      Will it help?
      Averaging time difference over long period of time will help, though one big delay will still create false positive.

      I think it would be better to implement this monitor as "simple check" within the server.

      server takes own servertimestamp1 then queries zabbix agnt for agenttimestamp, then server takes own servertimestamp2 again... Now we have 3 timestamps which look like this historically:

      servertimestamp1
      agenttimestamp
      servertimestamp2

      the corrected time delta between agent and server will be
      "agenttimestamp - (servertimestamp1+servertimestamp2)/2"

      The measuring error will be servertimestamp2 - servertimestamp1
      Now we can generate alarm with condition
      "if mod(time_delta) > (measuring_error + threshold_which_includes_rounding_error)"

      Also lets dont forget that servers can live in different time zones so timestamp sent should be in utime format (preferably hires utime).
      =========

      Also besides time measurement any metric can be timestamped on the agent (or in zabbix_sender) versus than on the server, then "change over time" graphs would look with less jitter because they wont have agent-to-server random lag.

      Comment

      • Alexei
        Founder, CEO
        Zabbix Certified Trainer
        Zabbix Certified SpecialistZabbix Certified Professional
        • Sep 2004
        • 5654

        #4
        Originally posted by alj
        Averaging time difference over long period of time will help, though one big delay will still create false positive.
        Why? The expression will be evaluated as '0' (problem) only in case of ALL values are out of the range.
        Alexei Vladishev
        Creator of Zabbix, Product manager
        New York | Tokyo | Riga
        My Twitter

        Comment

        • Alexei
          Founder, CEO
          Zabbix Certified Trainer
          Zabbix Certified SpecialistZabbix Certified Professional
          • Sep 2004
          • 5654

          #5
          Originally posted by alj
          Also besides time measurement any metric can be timestamped on the agent (or in zabbix_sender) versus than on the server, then "change over time" graphs would look with less jitter because they wont have agent-to-server random lag.
          ZABBIX cannot rely on agent timestamps! It would create many problems everywhere...
          Alexei Vladishev
          Creator of Zabbix, Product manager
          New York | Tokyo | Riga
          My Twitter

          Comment

          Working...