Ad Widget

Collapse

Extend loadable modules

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • kloczek
    Senior Member
    • Jun 2006
    • 1771

    #16
    Originally posted by vulogov
    Agree on that, but let it be a project for another daring soul. LUA is fine choice for small stuff (In My Humble Opinion), but if it worthy to employ "the language", you better use some more powerful one. Python/Ruby/Perl/CLIPS comes to mind. Me, personally, preferring OO programming for many reasons.
    Just entered into google search "zabbix lua" and I found this:


    !@#$ <censored> .. why I didn't this earlier?
    This looks like good sky fold for what I'm working on now
    http://uk.linkedin.com/pub/tomasz-k%...zko/6/940/430/
    https://kloczek.wordpress.com/
    zapish - Zabbix API SHell binding https://github.com/kloczek/zapish
    My zabbix templates https://github.com/kloczek/zabbix-templates

    Comment

    • mushero
      Senior Member
      • May 2010
      • 101

      #17
      As a company with dozens of custom bash scripts for Zabbix, and another few dozen agent changes pending, plus several SANs and other things that don't support SNMP, I can tell you this is VERY useful.

      We were ready to code lots of direct agent changes until the modules came along, so now we can use those, but still takes C programming skills which we have but in limited supply. Using Python makes more sense for a lot of what we do in many cases (for simple Linux stuff we'll probably still use modules).

      Many/most of our production systems are 16+ cores, dozens of GB of RAM, etc. so running a python script a few times every 300 seconds is not a serious challenge, given the advantage that I can actually get the monitoring, protocols, logic, and more that I want in real systems. Obviously one has to pay attention to the weight and effects, but valuable in simply enabling many things that were too hard before.

      There is a security element in that compiling things into the agent strikes me as safer than running easy-to-change python/ruby code on the server, i.e. I can control it better.

      But these are all just tools and options - modules are a huge improvement, and adding an external language like Ruby, Lua, Python, etc. is even better for many real-uses.

      Comment

      • vulogov
        Junior Member
        Zabbix Certified Specialist
        • Mar 2010
        • 14

        #18
        I am working on similar module, to extend not the agent, but the server. The module API are the same, but I think the protocol between C and Python shall be slightly different in the server case. Stay tuned.

        Comment

        • kloczek
          Senior Member
          • Jun 2006
          • 1771

          #19
          Originally posted by mushero
          But these are all just tools and options - modules are a huge improvement, and adding an external language like Ruby, Lua, Python, etc. is even better for many real-uses.
          Agree with this part.
          Still don't understand why something like SAN does not support SNMP and/or "why not SNMP?"
          http://uk.linkedin.com/pub/tomasz-k%...zko/6/940/430/
          https://kloczek.wordpress.com/
          zapish - Zabbix API SHell binding https://github.com/kloczek/zapish
          My zabbix templates https://github.com/kloczek/zabbix-templates

          Comment

          • mushero
            Senior Member
            • May 2010
            • 101

            #20
            Originally posted by kloczek
            Agree with this part.
            Still don't understand why something like SAN does not support SNMP and/or "why not SNMP?"
            It's an IBM SAN and believe me, we've tried - the management console will do some SNMP it seems, but the box itself will not - just proprietary stuff, and painful.

            But lots of other stuff we monitor like services, JMX, MySQL, HAProxy, custom apps, databases, 3rd party stuff, etc. that need these types of extensions.

            Comment

            • kloczek
              Senior Member
              • Jun 2006
              • 1771

              #21
              Originally posted by mushero
              It's an IBM SAN and believe me, we've tried - the management console will do some SNMP it seems, but the box itself will not - just proprietary stuff, and painful.

              But lots of other stuff we monitor like services, JMX, MySQL, HAProxy, custom apps, databases, 3rd party stuff, etc. that need these types of extensions.
              JMX data can be monitored over SNMP SMUX bridge. Zabbix supports own way of collectiong JMX data.
              Mysql does not need anything. I'm monitoring it without any backend scripts. Just mysql command without running additionally awk/grep/perl/etc. If you want I can share my own mysql template showing how to do this.
              HA Proxy IIRC has snmp SMUX/AgentX plugin.

              If your IBM SAN device exposes SAN stats over SNMP and IBM provides MIB for this device it should be no problem ..
              Last edited by kloczek; 04-11-2014, 04:06.
              http://uk.linkedin.com/pub/tomasz-k%...zko/6/940/430/
              https://kloczek.wordpress.com/
              zapish - Zabbix API SHell binding https://github.com/kloczek/zapish
              My zabbix templates https://github.com/kloczek/zabbix-templates

              Comment

              • mushero
                Senior Member
                • May 2010
                • 101

                #22
                Originally posted by kloczek
                JMX data can be monitored over SNMP SMUX bridge. Zabbix supports own way of collectiong JMX data.
                Mysql does not need anything. I'm monitoring it without any backend scripts. Just mysql command without running additionally awk/grep/perl/etc. If you want I can share my own mysql template showing how to do this.
                HA Proxy IIRC has snmp SMUX/AgentX plugin.

                If your IBM SAN device exposes SAN stats over SNMP and IBM provides MIB for this device it should be no problem ..
                Yes, Zabbix has a way to do JMX, but messy and ours is much better with direct access with no daemons or anything - we built it years ago and directly calls JMX from the agent/script.

                We also have a great mySQL interface we built that can get any variable, any status, parses error and innodb.status files; far better than any other tool or template we've seen. But happy to look at how you do it to learn more!

                Unfortunately, the SAN does not expose anything over SNMP that we can tell and IBM confirmed I think; you have to use their tools and GUI; seems absurd to me as it's new and high-end.

                Comment

                • kloczek
                  Senior Member
                  • Jun 2006
                  • 1771

                  #23
                  Originally posted by mushero
                  Yes, Zabbix has a way to do JMX, but messy and ours is much better with direct access with no daemons or anything - we built it years ago and directly calls JMX from the agent/script.

                  We also have a great mySQL interface we built that can get any variable, any status, parses error and innodb.status files; far better than any other tool or template we've seen. But happy to look at how you do it to learn more!

                  Unfortunately, the SAN does not expose anything over SNMP that we can tell and IBM confirmed I think; you have to use their tools and GUI; seems absurd to me as it's new and high-end.
                  In attachment you can find two templates: "Service MySQL" and "Service MySQL slave".
                  "Service MySQL" uses only system.run[] to execute mysql queries constructed in specific way to output only sampled data so filtering output of the mysql command is not needed.
                  On monitoring innodb mysql is good to change innodb_stats_on_metadata=0. It gives a little less accurate data but in the same time monitoring overhead and latency of the queries used by these templates are way smaller.
                  Attached Files
                  Last edited by kloczek; 04-11-2014, 06:41.
                  http://uk.linkedin.com/pub/tomasz-k%...zko/6/940/430/
                  https://kloczek.wordpress.com/
                  zapish - Zabbix API SHell binding https://github.com/kloczek/zapish
                  My zabbix templates https://github.com/kloczek/zabbix-templates

                  Comment

                  • mushero
                    Senior Member
                    • May 2010
                    • 101

                    #24
                    Thanks and interesting.

                    Note we can't really put our customers production passwords into Zabbix (too dangerous) and never allow/use system.run as too dangerous in our world - we run systems in 100+ data centers globally on untrusted links, and even then we can't expose our customers' systems to allowing Zabbix to run arbitrary commands - but very useful for more common single company LANs.

                    In our system the key is just something like mysql_realtime(bytes_sent) and we get the value. Far easier than having the huge string in each key, also more easily parsed for our DB integrations that read DB data directly for rule engines and predictive models, etc.

                    JMX works the same way, and runs on the host via the agent, just jmx(java.heap_max) sort of thing, pulls any JMX key with full security (which we setup when we enable JMX, with host/user/password in Java). Far, far simpler than what Zabbix's JMX gateway does and needs (not to mention security issues in distributed environments).

                    Comment

                    • kloczek
                      Senior Member
                      • Jun 2006
                      • 1771

                      #25
                      Originally posted by mushero
                      Thanks and interesting.

                      Note we can't really put our customers production passwords into Zabbix (too dangerous) and never allow/use system.run as too dangerous in our world - we run systems in 100+ data centers globally on untrusted links, and even then we can't expose our customers' systems to allowing Zabbix to run arbitrary commands - but very useful for more common single company LANs.
                      I understand this.
                      Mysql in exactly this case is terrible here because to monitor slave you must give almost full access to all databases. Using similar approach n case Oracle is possible to give very precise access to only engine metrics.
                      In this case it problem of the mysql.

                      In our system the key is just something like mysql_realtime(bytes_sent) and we get the value. Far easier than having the huge string in each key, also more easily parsed for our DB integrations that read DB data directly for rule engines and predictive models, etc.
                      It is only matter of abstraction of the queries to create shorter keys. I have some work done here. Only problem which I found with current modules API is that it is not possible to access to host/template macros within module extensions code.
                      My template only shows kind of low level way of accessing to mysql metrics in some very minimalists way reducing to minimum additional steps on processing output of the queries.
                      http://uk.linkedin.com/pub/tomasz-k%...zko/6/940/430/
                      https://kloczek.wordpress.com/
                      zapish - Zabbix API SHell binding https://github.com/kloczek/zapish
                      My zabbix templates https://github.com/kloczek/zabbix-templates

                      Comment

                      • mushero
                        Senior Member
                        • May 2010
                        • 101

                        #26
                        Originally posted by kloczek
                        I understand this.
                        Mysql in exactly this case is terrible here because to monitor slave you must give almost full access to all databases. Using similar approach n case Oracle is possible to give very precise access to only engine metrics.
                        In this case it problem of the mysql.
                        Not with 5.1 or later as can have very minimal user get all variables and status, so we use that. But we still can't centralize it as too dangerous for the customers and still security bugs in MySQL that make it unsafe to have a valid user. So our user/pw is local to the Zabbix user in .my.cnf on the server.

                        Comment

                        • kloczek
                          Senior Member
                          • Jun 2006
                          • 1771

                          #27
                          Originally posted by mushero
                          Not with 5.1 or later as can have very minimal user get all variables and status, so we use that. But we still can't centralize it as too dangerous for the customers and still security bugs in MySQL that make it unsafe to have a valid user. So our user/pw is local to the Zabbix user in .my.cnf on the server.
                          Still had no time to experiment with 5.5 and new auth interface which promises use kerberos, so it should be possible to specify that on host X for example zabbix user should be able to login in mysql without using password.
                          http://uk.linkedin.com/pub/tomasz-k%...zko/6/940/430/
                          https://kloczek.wordpress.com/
                          zapish - Zabbix API SHell binding https://github.com/kloczek/zapish
                          My zabbix templates https://github.com/kloczek/zabbix-templates

                          Comment

                          • LynxChaus
                            Junior Member
                            • Feb 2013
                            • 25

                            #28
                            Originally posted by kloczek
                            I wrote that (in ideal scenario) it would be good to have zabbix LUA module.
                            Where ? In server or in agent?

                            Comment

                            • kloczek
                              Senior Member
                              • Jun 2006
                              • 1771

                              #29
                              Originally posted by LynxChaus
                              Where ? In server or in agent?
                              Probably both.
                              http://uk.linkedin.com/pub/tomasz-k%...zko/6/940/430/
                              https://kloczek.wordpress.com/
                              zapish - Zabbix API SHell binding https://github.com/kloczek/zapish
                              My zabbix templates https://github.com/kloczek/zabbix-templates

                              Comment

                              • LynxChaus
                                Junior Member
                                • Feb 2013
                                • 25

                                #30
                                Originally posted by kloczek
                                Probably both.
                                For what zabbix version?

                                Comment

                                Working...