Ad Widget

Collapse

Better syslog message handling for Zabbix

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • jvandenbroek
    Junior Member
    • Oct 2011
    • 15

    #136
    I'm sorry and thanks for the solution. I really thought I had applied them all Working fine now!

    Comment

    • JBo
      Senior Member
      • Jan 2011
      • 310

      #137
      zbxlog release 1.2

      Hi,

      I have just released zbxlog v1.2 (http://www.alixen.org/attachments/do...bxlog-r1.2.tgz).

      This release includes Zabbix 1.8.6, 1.8.7 and 1.8.8 frontend patches.

      Happy syslog monitoring !
      JBo

      Comment

      • tiwi
        Junior Member
        • Nov 2011
        • 3

        #138
        I need an example how to use it

        as I'm absolutely new to Zabbix I think I need an example how to
        use zbxlog in Zabbix.

        Installation and starting is no problem but adding the syslog stuff to an existing host is a mystery for me.

        The README just says:

        Once zbxlog.pl is up and running, all configuration is done through Zabbix Web GUI.

        and now I'm lost ...

        Thank you very much
        Tiwi

        Comment

        • JBo
          Senior Member
          • Jan 2011
          • 310

          #139
          Hi tiwi,

          You need to add an item to your host.
          A simple item would be:
          • Type : Zabbix trapper
          • Type of information : Log
          • Key : syslog[]

          It will catch all syslog messages from your host.
          If you need to select only some types of messages, you can check «Usage» chapter in Zbxlog README file. It describes the syntax of syslog[] with examples.

          Hope this helps,
          JBo

          Comment

          • tiwi
            Junior Member
            • Nov 2011
            • 3

            #140
            Thank you very much, this did help me. Maby this should be included in the README file.


            Another question I got:
            I have installed zabbix 1.9.8 with zbxlog 1.2. I had to patch some files manually because the offset was too big.

            Now when I start zbxlog, I get this in the logfile:

            Prototype mismatch: sub Zbxlog::Controller::inet_ntoa ($) vs none at lib/Zbxlog/Controller.pm line 23

            is this something I should worry about?
            I'm asking because I'm not shure if anything is received from zbxlog.
            Is it possible to log incoming messages to the logfile somehow?

            Thanks
            Tiwi

            Comment

            • JBo
              Senior Member
              • Jan 2011
              • 310

              #141
              Hi,

              Originally posted by tiwi
              Thank you very much, this did help me. Maby this should be included in the README file.
              The example I gave comes straight from the README file, chapter «6. Usage»


              Originally posted by tiwi
              Another question I got:
              I have installed zabbix 1.9.8 with zbxlog 1.2. I had to patch some files manually because the offset was too big.
              I am not following 1.9 branch as closely as 1.8 because it is the development version.
              Updating the patches for 1.9 is on my TODO list

              Originally posted by tiwi
              Now when I start zbxlog, I get this in the logfile:

              Prototype mismatch: sub Zbxlog::Controller::inet_ntoa ($) vs none at lib/Zbxlog/Controller.pm line 23

              is this something I should worry about?
              I'm asking because I'm not shure if anything is received from zbxlog.
              No, it is just a warning.
              You can ignore it.

              Originally posted by tiwi
              Is it possible to log incoming messages to the logfile somehow?
              You can :
              1. create a default destination for all syslog messages in Zabbix so that all messages that don't match any defined item are catched. See «Default destination» section in README file for details.
              2. Each Perl class has its own debug flag. If you edit lib/Zbxlog/Controller.pm and set :
                Code:
                my $DEBUG = 1;
                at line 29, you should at least see all messages received by Zbxlog in zbxlog.log


              Hope this helps,
              JBo

              Comment

              • tiwi
                Junior Member
                • Nov 2011
                • 3

                #142
                Thanks a log, zbxlog is receiving messages now.

                For a zabbix newbe like me it was hard to understand the part 6 of the README. The concept of creating an item for a host was unclear.
                After your explanation and rereading it and creating a host entry with syslog, it obviously makes sense immediately.

                Thanks
                Tiwi

                Comment

                • zabbixflic
                  Member
                  • Jan 2012
                  • 39

                  #143
                  Hi people,

                  and first of all thnx for this great work. Syslog is a very important tool
                  as Zabbix, of course

                  I installed zbxlog correctly, it's running with no errors (checked log, only mismatch protocol to ignore) but on Zabbix via web I cannot
                  see syslog messages stored.

                  I tried to simulate using :

                  nc -w0 -u 192.168.0.1 514 <<< "testing again from my home machine"

                  logger "test test test test"

                  What can I do ? The only thing I did not make is patching cause my zabbix is 1.8.10.

                  Could you help me ?

                  Thank you so much in advance!

                  Federico

                  Comment

                  • Guschtl44
                    Junior Member
                    • Jan 2012
                    • 1

                    #144
                    Debian 6.0.3 / Zabbix 1.8.2. / zbxlog 1.2 Startup Error

                    Hello,

                    after setup and install all requirements i got the following error in the log

                    root@box:/var/log# tail -f /var/log/zbxlog.log
                    Fri Jan 13 22:19:37 2012 zbxlog.pl started
                    Prototype mismatch: sub Zbxlog::Controller::AF_INET6: none vs () at lib/Zbxlog/Controller.pm line 23
                    The process is showed

                    Code:
                    root@box:/var/log# ps aux|grep zbxl
                    root     13457  1.1  0.5  61040 10784 ?        S    22:19   0:05 /usr/bin/perl /usr/local/zbxlog/bin/zbxlog.pl
                    root     13553  0.0  0.0   9608   892 pts/0    S+   22:27   0:00 grep zbxl
                    and also the udp port is listening

                    root@box:/var/log# netstat -ulnp|grep 514
                    udp 0 0 0.0.0.0:514 0.0.0.0:* 13457/perl
                    there are many UDP pakets send to the server (testet by iptraf):

                    ¦ UDP (238 bytes) from syslog-source:2055 to zabbix-server:514 on eth0 ¦
                    ¦ UDP (137 bytes) from syslog-source:2055 to zabbix-server:514 on eth0 ¦
                    ¦ UDP (147 bytes) from syslog-source:2055 to zabbix-server:514 on eth0 ¦
                    ¦ UDP (116 bytes) from syslog-source:2055 to zabbix-server:514 on eth0
                    but i got no data to my zabbix item.

                    Pls. help me to get it working...

                    Many thanks!

                    Comment

                    • JBo
                      Senior Member
                      • Jan 2011
                      • 310

                      #145
                      Troubleshooting zbxlog

                      Hi Guschtl44 and zabbixflic,

                      You have both described what you've done on zbxlog side but no detail on what you did in Zabbix. I start this post as a general troubleshooting guide that may help you (and hopefully other Zbxlog users) and some answers to your specific questions.

                      In Zabbix, you should have at least:

                      1. An item set as:
                      • Type : Zabbix trapper
                      • Type of information : Log
                      • Key : syslog[]

                      on the host that should receive all syslog messages from this host.

                      2. An host named sysloghost with an item set as:
                      • Type : Zabbix trapper
                      • Type of information : Log
                      • Key : syslog_nomatch

                      as defined by parameters nomatch_host and nomatch_item in zbxlog.conf.
                      It will receive all syslog messages received by zbxlog that don't match any other parameter.

                      If syslog messages that should be received by the item set in (1) are actually received by syslog_nomatch (2), it means that zbxlog cannot determine zabbix host name based on IP address or DNS name (obtained by zbxlog by doing a reverse DNS resolution of IP address).
                      Syslog messages sent to syslog_nomatch are prepended with host IP address as seen by Zbxlog. It may help troubleshoot why mapping IP to zabbix host name fails.

                      If no syslog message is received by either items (1) or (2), zbxlog is probably not receiving any syslog message.
                      You can
                      • edit lib/Zbxlog/Controller.pm
                      • change
                        Code:
                        my $DEBUG = 0;
                        to
                        Code:
                        my $DEBUG = 1;
                      • save
                      • restart zbxlog service
                        Code:
                        /etc/init.d/zbxlog stop
                        /etc/init.d/zbxlog start

                      In /var/log/zbxlog.log you should see one line per syslog message received, such as:
                      Code:
                      Read:remoteip=192.168.0.100 remote_host=192.168.0.100 buf=<86>Jan 16 12:51:06 vega CRON[15425]: pam_unix(cron:session): session closed for user admin
                      If no line of this type appears in zbxlog.log, zbxlog is not receiving anything. netstat, tcpdump, iptraf are your friends

                      If you have this kind of lines but still nothing in zabbix, you can:
                      • edit lib/Zbxlog/Sender.pm
                      • change
                        Code:
                        my $DEBUG = 0;
                        to
                        Code:
                        my $DEBUG = 1;
                      • save
                      • restart zbxlog service

                      In /var/log/zbxlog.log you should see messages sent to zabbix, such as:
                      Code:
                      Zbxlog::Sender::Send item=$VAR1 = [
                                'vega',
                                '',
                                'syslog[]',
                                '86',
                                'authpriv',
                                11,
                                1326715565,
                                'CRON[15983]: pam_unix(cron:session): session closed for user admin'
                              ];
                      
                      Zbxlog::Sender::Send response=ZBXD^AW^@^@^@^@^@^@^@{
                              "response":"success",
                              "info":"Processed 1 Failed 0 Total 1 Seconds spent 0.000035"}
                      Check that response is success and Failed is 0. Otherwise, it means that Zabbix is rejecting this message. In this case, zabbix-server.log may contain useful error messages.

                      Some specific answers:
                      • Error message in zbxlog.log:

                      Code:
                      Prototype mismatch: sub Zbxlog::Controller::AF_INET6: none vs () at lib/Zbxlog/Controller.pm line 23
                      This is just a warning due to IO::Socket::INET6 Perl module. It can safely be ignored.
                      • syslog test message

                      Code:
                      nc -w0 -u 192.168.0.1 514 <<< "testing again from my home machine"
                      Syslog messages should be RFC compliant otherwise, Zbxlog will discard them.
                      Code:
                      <86>Jan 16 12:51:06 vega CRON[15425]: pam_unix(cron:session): session closed for user admin
                      Is OK.

                      • Patches for Zabbix frontend.

                      Patches for 1.8.5 can be applied to 1.8.10

                      Hope this helps,
                      JBo

                      Comment

                      • zabbixflic
                        Member
                        • Jan 2012
                        • 39

                        #146
                        Hi JBo,

                        and many thanks for your Help.

                        First of all, now it works you're great !

                        Second :

                        - I receive all messages on sysloghost -> I created an host
                        called "syslogfiltered" (with 0.0.0.0 as ip address) and add an item
                        as written at point 1 but I cannot receive messages on that; also
                        sysloghost ip address is set to 0.0.0.0

                        - patching files I obtained :

                        /usr/local/zbxlog/zabbix/patches/1.8.5/frontends/php/en_gb.inc.php.patch
                        patching file include/locales/en_gb.inc.php
                        Hunk #1 succeeded at 1054 (offset 5 lines).
                        root@ubuntu:/home/zabbix/public_html# patch -p0 <

                        /usr/local/zbxlog/zabbix/patches/1.8.5/frontends/php/history.php.patch
                        patching file history.php
                        Hunk #1 succeeded at 318 (offset 8 lines).

                        Remaining files are OK.

                        - which is the best way to see collected messages ? I see them on "Latest data -> sysloghost -> zbxlog (my item name) -> history
                        but I am not sure this is right. Can we see them in event in such a way, in order to use ack, severity etc?

                        Many thanks again

                        Federico

                        Comment

                        • JBo
                          Senior Member
                          • Jan 2011
                          • 310

                          #147
                          Originally posted by zabbixflic
                          Hi JBo,

                          and many thanks for your Help.

                          First of all, now it works you're great !
                          Good news !

                          Originally posted by zabbixflic

                          Second :

                          - I receive all messages on sysloghost -> I created an host
                          called "syslogfiltered" (with 0.0.0.0 as ip address) and add an item
                          as written at point 1 but I cannot receive messages on that; also
                          sysloghost ip address is set to 0.0.0.0
                          Zbxlog uses the IP address of the host sending suslog messages to find corresponding zabbix host. 0.0.0.0 will never match a real IP.
                          As I said in previous post, zbxlog does also a reverse DNS lookup based on the IP. If you set the IP in Zabbix to 0.0.0.0, reverse DNS lookup MUST match the "DNS name" set in Zabbix.

                          Originally posted by zabbixflic
                          - patching files I obtained :

                          /usr/local/zbxlog/zabbix/patches/1.8.5/frontends/php/en_gb.inc.php.patch
                          patching file include/locales/en_gb.inc.php
                          Hunk #1 succeeded at 1054 (offset 5 lines).
                          root@ubuntu:/home/zabbix/public_html# patch -p0 <

                          /usr/local/zbxlog/zabbix/patches/1.8.5/frontends/php/history.php.patch
                          patching file history.php
                          Hunk #1 succeeded at 318 (offset 8 lines).

                          Remaining files are OK.
                          There is a slight difference between 1.8.5 and 1.8.10 but all patches still apply.
                          Code:
                          Hunk #1 succeeded at 318
                          is just a warning.

                          Originally posted by zabbixflic

                          - which is the best way to see collected messages ? I see them on "Latest data -> sysloghost -> zbxlog (my item name) -> history
                          That's the best way.

                          Originally posted by zabbixflic
                          but I am not sure this is right. Can we see them in event in such a way, in order to use ack, severity etc?
                          All those syslog messages are just Zabbix items. If you want to deal with alerts with severity, ack,... you need to define triggers. There are a few examples in README. You'll get more detailed explanations on triggers in Zabbix documentation. All functions that apply to "log" type items also apply to syslog messages.


                          Hope this helps,
                          JBo

                          Comment

                          • zabbixflic
                            Member
                            • Jan 2012
                            • 39

                            #148
                            Originally posted by JBo
                            Good news !

                            Zbxlog uses the IP address of the host sending suslog messages to find corresponding zabbix host. 0.0.0.0 will never match a real IP.
                            As I said in previous post, zbxlog does also a reverse DNS lookup based on the IP. If you set the IP in Zabbix to 0.0.0.0, reverse DNS lookup MUST match the "DNS name" set in Zabbix.
                            JBo
                            ok, but I have also 1 item created with 10.129.93.154 that is real ip address
                            of Zabbix server. I don't understand why it cannot collect data

                            Originally posted by JBo
                            you need to define triggers. There are a few examples in README. You'll get more detailed explanations on triggers in Zabbix documentation. All functions that apply to "log" type items also apply to syslog messages.


                            Hope this helps,
                            JBo
                            I supposed. Good !

                            Comment

                            • zabbixflic
                              Member
                              • Jan 2012
                              • 39

                              #149
                              Well, activating debug I understood.

                              I cannot specify ip address of Zabbix server to aggregate all syslog messages and filter using [], I must create "n" Items for "n" hosts with correct "source" syslog messages ip address.

                              Am I right ?

                              Federico

                              Comment

                              • JBo
                                Senior Member
                                • Jan 2011
                                • 310

                                #150
                                Originally posted by zabbixflic
                                Well, activating debug I understood.

                                I cannot specify ip address of Zabbix server to aggregate all syslog messages and filter using [], I must create "n" Items for "n" hosts with correct "source" syslog messages ip address.

                                Am I right ?
                                Almost, yes.

                                zbxlog follows Zabbix philosophy: syslog messages are items and, as such, are attached to Zabbix hosts.
                                If you have syslog messages coming from N different hosts, you should create N Zabbix hosts, each one with its own IP and at least one syslog[] item.

                                JBo

                                Comment

                                Working...