Ad Widget

Collapse

All graphs stopped working

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • fleon
    Junior Member
    • Aug 2014
    • 22

    #1

    All graphs stopped working

    We had a power issue and when it got resolved no single graph appears to get data. I don't get errors on zabbix_server.log and maps work with simple triggers (icmp) but all graphs appear blank. Is there some log i can check? I tried deleting a host and importing it again , and rebooting again, i can see it being monitored but i am waiting to see if the discovery gets triggered so i can monitor traffic on the interfaces.
  • aib
    Senior Member
    • Jan 2014
    • 1615

    #2
    Is your database works?
    Usually after power outage the database daemon have a hard time to restart.

    Check /var/log/mysql.log or any log-files of your DB daemon.
    Sincerely yours,
    Aleksey

    Comment

    • fleon
      Junior Member
      • Aug 2014
      • 22

      #3
      the database is up otherwise the system would not even work.
      i am using debian stable and all mysql logs are empty and i see it running on top

      using \s on the mysql console gives:

      Threads: 24 Questions: 62702 Slow queries: 177 Opens: 669 Flush tables: 1 Open tables: 215 Queries per second avg: 19.606

      So i don't think the DB is to blame, although the server was painfully slow after restart, so i rebooted it again (just before posting) and am waiting for the 1 hour uptime to see if the discovery will run.

      I don't want to delete the hosts and import them again, because that will mess my whole map

      Comment

      • fleon
        Junior Member
        • Aug 2014
        • 22

        #4
        On administration->queue i see the following:

        SNMPv2 agent 0 0 0 0 0 23009

        There are over 23009 items that the server needs to update and it isn't doing so, i don't understand why i am not seeing something on the logs.

        For example, on one host it updated the values as soon as i rebooted (11:35am) but it hasn't attempted to try again, there is something definitely wrong.

        Comment

        • fleon
          Junior Member
          • Aug 2014
          • 22

          #5
          Code:
          UID        PID  PPID  C STIME TTY          TIME CMD
          zabbix    3149     1  0 11:34 ?        00:00:00 /usr/sbin/zabbix_server
          zabbix    3177  3149  0 11:35 ?        00:00:11 /usr/sbin/zabbix_server: configuration syncer [synced configuration in 1.158377 sec, idle 60 sec]
          zabbix    3178  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: db watchdog [synced alerts config in 0.000424 sec, idle 60 sec]
          zabbix    3179  3149  0 11:35 ?        00:00:03 /usr/sbin/zabbix_server: poller #1 [got 273 values in 1929.151966 sec, getting values]
          zabbix    3180  3149  0 11:35 ?        00:00:03 /usr/sbin/zabbix_server: poller #2 [got 502 values in 1929.594968 sec, getting values]
          zabbix    3181  3149  0 11:35 ?        00:00:03 /usr/sbin/zabbix_server: poller #3 [got 172 values in 1929.245902 sec, getting values]
          zabbix    3182  3149  0 11:35 ?        00:00:03 /usr/sbin/zabbix_server: poller #4 [got 195 values in 1929.348425 sec, getting values]
          zabbix    3183  3149  0 11:35 ?        00:00:03 /usr/sbin/zabbix_server: poller #5 [got 206 values in 1929.255203 sec, getting values]
          [B]zabbix    3184  3149  0 11:35 ?        00:00:01 /usr/sbin/zabbix_server: unreachable poller #1 [got 0 values in 0.000006 sec, idle 5 sec][/B]
          zabbix    3185  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: trapper #1 [processed data in 0.000447 sec, waiting for connection]
          zabbix    3186  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: trapper #2 [processed data in 0.000395 sec, waiting for connection]
          zabbix    3187  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: trapper #3 [processed data in 0.000481 sec, waiting for connection]
          zabbix    3188  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: trapper #4 [processed data in 0.000581 sec, waiting for connection]
          zabbix    3189  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: trapper #5 [processed data in 0.000421 sec, waiting for connection]
          zabbix    3190  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: icmp pinger #1 [pinging hosts]
          zabbix    3191  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: alerter [sent alerts: 0 success, 0 fail in 0.000115 sec, idle 30 sec]
          zabbix    3192  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: housekeeper [removing old history and trends]
          zabbix    3193  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: timer #1 [processed 0 triggers, 0 events in 0.000000 sec, 0 maintenances in 0.000000 sec,
          zabbix    3194  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: http poller #1 [got 0 values in 0.000501 sec, idle 5 sec]
          zabbix    3195  3149  0 11:35 ?        00:00:01 /usr/sbin/zabbix_server: discoverer #1 [processed 0 rules in 0.000719 sec, idle 60 sec]
          zabbix    3196  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: history syncer #1 [synced 3126 items in 53.572267 sec, syncing history]
          zabbix    3197  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: history syncer #2 [synced 2977 items in 60.508110 sec, syncing history]
          zabbix    3198  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: history syncer #3 [synced 3299 items in 64.675326 sec, syncing history]
          zabbix    3199  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: history syncer #4 [synced 71 items in 13.172267 sec, syncing history]
          zabbix    3200  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: escalator [processed 0 escalations in 0.000119 sec, idle 3 sec]
          zabbix    3201  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: proxy poller #1 [exchanged data with 0 proxies in 0.000005 sec, idle 5 sec]
          zabbix    3202  3149  0 11:35 ?        00:00:00 /usr/sbin/zabbix_server: self-monitoring [processed data in 0.000008 sec, idle 1 sec]
          This is the process info for zabbix, is that unreachable message for the poller supposed to be normal?

          Comment

          • fleon
            Junior Member
            • Aug 2014
            • 22

            #6
            well i just found out that if i restart the zabbix service, a batch of items get updated and it stops updating again.

            i am very close of taking the decision to delete the whole thing and start again, at least this time i have saved the hosts , templates and group definitions.

            Comment

            • fleon
              Junior Member
              • Aug 2014
              • 22

              #7
              suddenly at 5pm zabbix started working again, as if i had some kind of ghost maintenance without data collection. i was very close to deleting the whole database and start again

              Comment

              Working...