Ad Widget

Collapse

We're getting Trigger Alerts when we should not

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • just2blue4u
    Senior Member
    • Apr 2006
    • 347

    #1

    We're getting Trigger Alerts when we should not

    Hi there!

    We keep getting alerts (and upalerts) for systems that have been switched off since Friday.

    They say "CPU Load too high" or "Zabbix agent unreachable for over 2 minutes", or "Zabbix agend reachable again" (as said- that system is still switched off).

    System Load of our zabbix server is ok, the mysql-db seems happy, too.
    I restarted the zabbix server program but that didn't help!
    When i see the history of actions, all Alerts are listed as "not sent" with 3 retries left. We do send them via email and windows popup (selfmade shellscript using samba).

    We do get alerts, so mail and script do work good, but why is zabbix sending us this crap? A switched off system can't switch between reachable and unreachable all the time; it should be unreachable all the time!

    What can i do to get rid of that alerts?
    Big ZABBIX is watching you!
    (... and my 48 hosts, 4513 items, 1280 triggers via zabbix v1.6 on CentOS 5.0)
  • richlv
    Senior Member
    Zabbix Certified Trainer
    Zabbix Certified SpecialistZabbix Certified Professional
    • Oct 2005
    • 3112

    #2
    Originally posted by just2blue4u
    ...
    When i see the history of actions, all Alerts are listed as "not sent" with 3 retries left. We do send them via email and windows popup (selfmade shellscript using samba).
    could it be that your method of sending alerts does not return correct exit code, so zabbix assumes the message was not sent successfully ?
    Zabbix 3.0 Network Monitoring book

    Comment

    • just2blue4u
      Senior Member
      • Apr 2006
      • 347

      #3
      since when does Zabbix try to re-send unsent alerts ?

      We send via email and with this script:
      Code:
      #!/bin/bash
      #
      # Send a zabbix message via winpopup to a given host
      # Use as an alertscript within zabbix
      # Syntax: winpopup $receiptant $subject $message
      # Note: zabbix passes $1=receiptiant, $2=subject, $3=message to any script
      echo -e "=============================================================\\n$2\\n=============================================================\\n\\n$3"  | smbclient -M $1  -U="zabbix imv neu"
      Today this problems occured first. We didn't do anything with our server (no update of zabbix server bins)
      Big ZABBIX is watching you!
      (... and my 48 hosts, 4513 items, 1280 triggers via zabbix v1.6 on CentOS 5.0)

      Comment

      • richlv
        Senior Member
        Zabbix Certified Trainer
        Zabbix Certified SpecialistZabbix Certified Professional
        • Oct 2005
        • 3112

        #4
        Originally posted by just2blue4u
        since when does Zabbix try to re-send unsent alerts ?
        i haven't investigated this, i don't remember having failed sending attempts...
        Originally posted by just2blue4u
        We send via email and with this script:
        ...
        try to run the script as zabbix user with the exact parameters that are used in problematic action.
        Zabbix 3.0 Network Monitoring book

        Comment

        • just2blue4u
          Senior Member
          • Apr 2006
          • 347

          #5
          that problem stopped and didn't come again.
          Seems like it was the server being confused after a server and db restart.

          Is this a bug or was it just bad luck?
          Big ZABBIX is watching you!
          (... and my 48 hosts, 4513 items, 1280 triggers via zabbix v1.6 on CentOS 5.0)

          Comment

          • richlv
            Senior Member
            Zabbix Certified Trainer
            Zabbix Certified SpecialistZabbix Certified Professional
            • Oct 2005
            • 3112

            #6
            do these triggers have 'nodata' in their expressions ?
            Zabbix 3.0 Network Monitoring book

            Comment

            • just2blue4u
              Senior Member
              • Apr 2006
              • 347

              #7
              yes:
              {<host>roc.num[zabbix_agentd].nodata(120)}
              Big ZABBIX is watching you!
              (... and my 48 hosts, 4513 items, 1280 triggers via zabbix v1.6 on CentOS 5.0)

              Comment

              • richlv
                Senior Member
                Zabbix Certified Trainer
                Zabbix Certified SpecialistZabbix Certified Professional
                • Oct 2005
                • 3112

                #8
                which zabbix version ?
                there was a bug involving nodata triggers and zabbix server restarts, which is supposed to be fixed in 1.4.5 (though i still got the problem once after upgrading to 1.4.5 - kinda unwilling to reproduce, having several hundred warning emails sent around was bad once...)
                Zabbix 3.0 Network Monitoring book

                Comment

                • just2blue4u
                  Senior Member
                  • Apr 2006
                  • 347

                  #9
                  Server Version is 1.4.4 (just updated my sig! ). Clients are different...
                  Big ZABBIX is watching you!
                  (... and my 48 hosts, 4513 items, 1280 triggers via zabbix v1.6 on CentOS 5.0)

                  Comment

                  • richlv
                    Senior Member
                    Zabbix Certified Trainer
                    Zabbix Certified SpecialistZabbix Certified Professional
                    • Oct 2005
                    • 3112

                    #10
                    would be useful to check out whether you see this behaviour with 1.4.5 as well.
                    Zabbix 3.0 Network Monitoring book

                    Comment

                    • just2blue4u
                      Senior Member
                      • Apr 2006
                      • 347

                      #11
                      i'm writing an insttallscript for clientinstallation on rpm-based dell machines.
                      Planned to do the Server update after that!
                      Big ZABBIX is watching you!
                      (... and my 48 hosts, 4513 items, 1280 triggers via zabbix v1.6 on CentOS 5.0)

                      Comment

                      Working...