Ad Widget

Collapse

interruption on zabbix server

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • bbrendon
    Senior Member
    • Sep 2005
    • 870

    #31
    Originally posted by tchjts1
    It appears to be on the MySql side of the setup.
    I have separate Zabbix App and DB servers.
    Damn I'm good. Did I call that or what?
    Unofficial Zabbix Expert
    Blog, Corporate Site

    Comment

    • tchjts1
      Senior Member
      • May 2008
      • 1605

      #32
      Originally posted by bbrendon
      Damn I'm good. Did I call that or what?
      I would wholeheartedly agree with that if you would have come over and found exactly what the cause was and fixed it.

      I think part of the problem was that I did a fresh App install and only dropped the database on the DB server, but left mysql.sql behind. It was still picking up some olde hosts names and trying to do active checks with them even though they were no longer in the frontend.

      Hindsight is 20/40

      Comment

      • bbrendon
        Senior Member
        • Sep 2005
        • 870

        #33
        Hmmm, what you said doesn't quite make sense to me, but I'm glad you're making progress.

        Dropping the database, reimporting the schema, and restarting zabbix_server = basically a fresh install
        Unofficial Zabbix Expert
        Blog, Corporate Site

        Comment

        • alucard95
          Member
          • Sep 2008
          • 38

          #34
          This solution is not valid for me. I try differents architectures:

          - app server and db server on the same machine.

          - Dump of zabbix database, and new installation on two machines (app and db)

          - A new install on an other server with a new database (only few hosts)

          Maybe the problem is with a configuration but I don't know which one...

          PS: I test with MyISAM and innoDB engine

          Comment

          • alucard95
            Member
            • Sep 2008
            • 38

            #35
            Warning. Incorrect value for [period]

            Hi all,

            To bypass my problem with the graphs, I change history value.

            In defines.inc.php:

            Code:
            define('ZBX_MIN_PERIOD', 3600);                                       // 1 hour

            to
            Code:
            define('ZBX_MIN_PERIOD', 6*3600);                                       // 6 heures
            But now, when I want to create a graph, I have this message:
            Code:
            Warning. Incorrect value for [period]
            Where can I define value [period]

            Thanks

            Comment

            • MrKen
              Senior Member
              • Oct 2008
              • 652

              #36
              Hi Alucard95,

              It's starting to look like you are running around trying to chase your own tail, and getting nowhere!

              I really don't think that changing the graph MIN_PERIOD is going to solve your problem.

              Earlier you posted this, quote: "Sometimes, there is "hole" in my graph although I still have data."
              The 'I still have data' is relevant. During the graph holes period, is the latest data --> Values collected normally, I mean like every X seconds as scheduled, or is it collected sporadically. If it is sporadic, then that may account for the holes.

              Some good advice from bbrendon was, quote: "you could try increasing the number of trapper and snmp processes (any maybe others) in zabbix_server.conf"

              Considering that you have just about tried everything, I would try tweaking zabbix_server.conf.
              After a day or so if nothing changes, next I would try tweaking my.cnf by using the suggestions made by such scripts as mysqltuner.pl and tuning-primer.sh

              Also, I think that your problem is not the same as tchjts1, because he is only monitoring 2 hosts so therefore there is little or no load on Zabbix.

              But, I don't think it has anything to do with Zabbix being overloaded. I have 3 times as many items and 4 times as many triggers as you have, but I have Zabbix and Mysql running on one box with only 2Gb of RAM! [Required server performance, new values per second: 55.6142]
              BTW, I rarely have anything in the Queue, ever!

              HTH MrKen
              Disclaimer: All of the above is pure speculation.

              Comment

              • alucard95
                Member
                • Sep 2008
                • 38

                #37
                Hi MrKen,

                Changing the MIN_PERIOD allow to disappear the hole, but I agree with you, It's not the good solution, just temporary.


                Earlier you posted this, quote: "Sometimes, there is "hole" in my graph although I still have data."
                The 'I still have data' is relevant. During the graph holes period, is the latest data --> Values collected normally, I mean like every X seconds as scheduled, or is it collected sporadically. If it is sporadic, then that may account for the holes.
                To Illustrate:

                today, a graph for one item:



                We can see the hole nearly 8:40 am (for the example)

                To check the data, I check the history for this item in the database:

                ITEMID ============================ CLOCK ======================================== VALUE



                So there are data in this intervale but maybe not enough, effectively my item has an update intervale to 30s.

                Comment

                • MrKen
                  Senior Member
                  • Oct 2008
                  • 652

                  #38
                  alucard95,

                  On your Monitoring -> Latest Data - Graph page, at the top right there is a dropdown box. Choose 'Values'. Have a look at the values from around 8.30 to 8.50 and see if the values are updated every 30 seconds. For that 20 minute period there should be 40 values. [A bit easier than trying to understand the clock time from the database].
                  Disclaimer: All of the above is pure speculation.

                  Comment

                  • alucard95
                    Member
                    • Sep 2008
                    • 38

                    #39
                    Ok, it is easier than check in the database

                    So, this is the result:
                    Code:
                    2009-05-06 08:50:56	1241592656	576103.1724
                    2009-05-06 08:48:31	1241592511	400395.9255
                    2009-05-06 08:45:50	1241592350	405633.2818
                    2009-05-06 08:42:49	1241592169	424585.6495
                    2009-05-06 08:39:35	1241591975	172642.2642
                    2009-05-06 08:36:56	1241591816	255105.1095
                    2009-05-06 08:34:39	1241591679	186496.5556
                    2009-05-06 08:33:27	1241591607	103458.8571
                    2009-05-06 08:32:45	1241591565	159558.1818
                    2009-05-06 08:32:01	1241591521	161921.6667
                    2009-05-06 08:31:13	1241591473	720103.5429
                    2009-05-06 08:30:38	1241591438	175075.2000
                    There are effectively not enough value, only 12...

                    Here the configuration of the item:

                    Comment

                    • MrKen
                      Senior Member
                      • Oct 2008
                      • 652

                      #40
                      Not enough data = holes in graphs!

                      Your item configuration looks fine. I would start by increasing Trappers and Pollers in zabbix_server.conf
                      Increase by how much? Just guessing, but maybe increase to 30 or 40 and see what happens.

                      Good luck
                      MrKen
                      Disclaimer: All of the above is pure speculation.

                      Comment

                      • alucard95
                        Member
                        • Sep 2008
                        • 38

                        #41
                        Good news, it appears to be the problem.

                        I increase Zabbix trappers (default value 50):

                        Code:
                        StartTrappers=100
                        and next, the number of value increase too.

                        Code:
                        2009-05-06 12:50:17	1241607017	922617.3023
                        2009-05-06 12:49:34	1241606974	1011657.7021
                        2009-05-06 12:48:47	1241606927	869391.1795
                        2009-05-06 12:48:08	1241606888	782235.9184
                        2009-05-06 12:47:19	1241606839	963261.7674
                        2009-05-06 12:46:36	1241606796	1070118.8293
                        2009-05-06 12:45:55	1241606755	805330.6667
                        2009-05-06 12:45:13	1241606713	821700.6316
                        2009-05-06 12:44:35	1241606675	1034401.9130
                        2009-05-06 12:43:49	1241606629	1292326.0488
                        2009-05-06 12:43:08	1241606588	749623.4545
                        2009-05-06 12:42:24	1241606544	1163448.9524
                        2009-05-06 12:41:42	1241606502	1201734.4762
                        2009-05-06 12:41:00	1241606460	1004761.3023
                        2009-05-06 12:40:17	1241606417	926177.3333
                        2009-05-06 12:39:35	1241606375	2409126.4000
                        2009-05-06 12:38:50	1241606330	908475.8182
                        2009-05-06 12:38:06	1241606286	792037.5814
                        2009-05-06 12:37:23	1241606243	828074.0426
                        2009-05-06 12:36:36	1241606196	942758.8837
                        2009-05-06 12:35:53	1241606153	678907.3778
                        2009-05-06 12:35:08	1241606108	781302.7826
                        2009-05-06 12:34:22	1241606062	684770.4000
                        2009-05-06 12:33:42	1241606022	1103674.4000
                        2009-05-06 12:32:52	1241605972	654777.0256
                        2009-05-06 12:32:13	1241605933	693400.6154
                        2009-05-06 12:31:34	1241605894	616564.6667
                        2009-05-06 12:30:46	1241605846	632178.1463
                        2009-05-06 12:30:05	1241605805	581379.0000
                        And during 2 hours no hole until 13h, I check values and I see this:
                        Code:
                        2009-05-06 13:10:27	1241608227	910728.5634
                        2009-05-06 13:09:16	1241608156	892721.2308
                        2009-05-06 13:07:58	1241608078	963678.3273
                        2009-05-06 13:06:08	1241607968	872105.0714
                        2009-05-06 13:04:16	1241607856	1020059.7867
                        2009-05-06 13:01:46	1241607706	920117.8667
                        2009-05-06 12:59:01	1241607541	843810.7481
                        2009-05-06 12:56:50	1241607410	897456.3299
                        2009-05-06 12:55:13	1241607313	1009464.1951
                        Here, the number of value decrease, there is no error in zabbix_server.log.

                        I will try to increase trapper to 150 and I tell you the result.
                        Last edited by alucard95; 06-05-2009, 13:53.

                        Comment

                        • alucard95
                          Member
                          • Sep 2008
                          • 38

                          #42
                          Hi,

                          Some news since the last time. I upgrade on version 1.6.6 this morning and the gap practically disappear .
                          I only had 2 gaps since 3 hours and they correspond at Executing housekeeper in my zabbix_server.log

                          Can i suppose that during the cleaning my database server is overload?

                          Comment

                          • alucard95
                            Member
                            • Sep 2008
                            • 38

                            #43
                            I confirm that the "gaps" appears during the "Executing housekeeper" which overload my Database server (MySQL).

                            If I disable the housekeeper, there is no gap .

                            So I reduce the frequency of housekeeper and i'm going to try to optimize MySQL server.

                            Thank you for your helps

                            Comment

                            • Alexei
                              Founder, CEO
                              Zabbix Certified Trainer
                              Zabbix Certified SpecialistZabbix Certified Professional
                              • Sep 2004
                              • 5654

                              #44
                              Originally posted by alucard95
                              I confirm that the "gaps" appears during the "Executing housekeeper" which overload my Database server (MySQL).

                              If I disable the housekeeper, there is no gap .

                              So I reduce the frequency of housekeeper and i'm going to try to optimize MySQL server.

                              Thank you for your helps
                              A brief HOWTO on disabling housekeeper:

                              - upgrade to MySQL 5.1.x for support of table partitions
                              - partition historical (history*, events, trends*, other large tables) tables based on record timestamps
                              - create a crontab, which would create and drop partitions automatically
                              - disable housekeeper

                              Database partitions (pros and cons):

                              + no need to run housekeeper
                              + better performance of the server and GUI
                              + database size is nicely controlled
                              + dropping older data in a matter of seconds
                              - requires extra setup and maintenance work
                              Alexei Vladishev
                              Creator of Zabbix, Product manager
                              New York | Tokyo | Riga
                              My Twitter

                              Comment

                              • alucard95
                                Member
                                • Sep 2008
                                • 38

                                #45
                                Thanks Alexei for this howto,

                                I'm going to try it and I give you some feedbacks

                                Comment

                                Working...