Ad Widget

Collapse

Better syslog message handling for Zabbix

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • JBo
    Senior Member
    • Jan 2011
    • 310

    #151
    zbxlog release 1.3

    Hi,

    I have just released zbxlog v1.3 (http://www.alixen.org/attachments/do...bxlog-r1.3.tgz).

    This release includes Zabbix 1.8.10 and 1.9.9 frontend patches.

    Happy syslog monitoring !
    JBo

    Comment

    • zabbixflic
      Member
      • Jan 2012
      • 39

      #152
      Many thanks JBO.

      I love this tool !

      I think the only difference is patching other zabbix version,
      isn't it ?

      So, if I am not wrong, I must only overwrite previous patched files.

      Can you confirm ?

      Thnx again

      F.

      Comment

      • JBo
        Senior Member
        • Jan 2011
        • 310

        #153
        Hi

        Originally posted by zabbixflic
        I think the only difference is patching other zabbix version,
        isn't it ?
        That's right.

        Originally posted by zabbixflic
        So, if I am not wrong, I must only overwrite previous patched files.

        Can you confirm ?
        If you already have a working zbxlog installation, no need to upgrade.
        This new version only contain patches that apply cleanly to latest Zabbix frontend PHP files.

        Regards,
        JBo

        Comment

        • JBo
          Senior Member
          • Jan 2011
          • 310

          #154
          zbxlog release 1.4

          Hi,

          I have just released zbxlog v1.4 (http://www.alixen.org/attachments/do...bxlog-r1.4.tgz).

          This release includes Zabbix 2.0.0 frontend patches.

          Happy syslog monitoring !
          JBo

          Comment

          • BDiE8VNy
            Senior Member
            • Apr 2010
            • 680

            #155
            Using %fromhost-ip% does not work

            Hi!

            First I want to say that this is a pretty cool piece of software!

            My hosts have set their FQDN as hostname. (r)Syslog puts the short hostname into the %hostname% field. Therefore I want to use the %fromhost-ip% as reference but that doesn't work.

            Using the template definition:
            Code:
            $template ZBXLOG,"%HOSTNAME%.example.com%rawmsg%\n"
            works as expected. But using:
            Code:
            $template ZBXLOG,"%fromhost-ip%%rawmsg%\n"
            doesn't.

            Setting the DEBUG flag shows that the correct ip address is used.

            The host configuration in Zabbix has the following settings:
            Name: <FQDN>
            DNS name: <FQDN>
            IP address: <IPADDR>
            Connect to: "IP address"

            Comment

            • parcival
              Member
              • Sep 2010
              • 44

              #156
              Hi all,
              Anyone successful with Zabbix 2.0.4 ?

              thx
              parcival

              Comment

              • JBo
                Senior Member
                • Jan 2011
                • 310

                #157
                zbxlog release 1.5

                Hi,

                I have just released zbxlog v1.5 (http://www.alixen.org/attachments/do...bxlog-r1.5.tgz).

                This release includes Zabbix 2.0.4 frontend patches.

                Happy syslog monitoring !
                JBo

                Comment

                • parcival
                  Member
                  • Sep 2010
                  • 44

                  #158
                  Hi JBo,
                  big thank you for your fast answer
                  But i get no data in the database with zbxlog, although the zbxlog sender says it is all OK.

                  Please look here:
                  Code:
                  Zbxlog::Sender::Send item=$VAR1 = [
                            'fwgate-1',
                            '',
                            'syslog[]',
                            '130',
                            'local0',
                            15,
                            1360073792,
                            'fwgate-1: NetScreen device_id=fwgate-1  [Root]system-critical-00430: Dst IP session limit! From 123.XXX.XXX.XXX:55482 to 217.XXX.XXX.XXX:53, p
                  roto UDP (zone Untrust int  ethernet0/0). Occurred 1 times. (2013-02-05 15:16:31).'
                          ];
                  
                  Zbxlog::Sender::Send response=OK
                  any idea?
                  parcival

                  Comment

                  • parcival
                    Member
                    • Sep 2010
                    • 44

                    #159
                    Hi, and i get this with debug zabbix_server.
                    I missing here facilities and severities value.

                    Code:
                    42899:20130205:160752.352 Trapper got [{"request":"sender data",
                    "data":[
                    {
                    "host":"fwgate-1",
                    "key":"syslog[]",
                    "value":"fwgate-1: NetScreen device_id=fwgate-1  [Root]system-critical-00430: Dst IP session limit! From 176.XXX.XX.X:17883 to 217.XXX.XXX.XXX
                    7:53, proto UDP (zone Untrust int  ethernet0/0). Occurred 9 times. (2013-02-05 16:07:51)] len 299
                    but the zbxlog send this:
                    Code:
                    Zbxlog::Sender::Send item=$VAR1 = [
                              'fwgate-1',
                              '',
                              'syslog[]',
                              '130',
                              'local0',
                              15,
                              1360072630,
                              'fwgate-1: NetScreen device_id=fwgate-1  [Root]system-critical-00430: Dst IP session limit! From 176.XXX.XXX.X:57229 to 217.XXX.XXX.
                    .XXX:53, proto UDP (zone Untrust int  ethernet0/0). Occurred 15 times. (2013-02-05 14:57:09).'
                            ];
                    what is wrong here ?

                    thx
                    parcival

                    Comment

                    • JBo
                      Senior Member
                      • Jan 2011
                      • 310

                      #160
                      Hi parcival,

                      Sorry but I can't reproduce your problem.

                      According to the zabbix_server log, it appears that the message is truncated: there is no ending quote after '(2013-02-05 16:07:51)' and other fields are missing.

                      I have suspected that the whole message has been tuncated and I have tried with logger to send a very long message.

                      This is what I get in zbxlog.log:
                      Code:
                      Zbxlog::Sender::Send item=$VAR1 = [
                                'Zabbix server',
                                '',
                                'syslog[]',
                                '13',
                                'user',
                                12,
                                1360154656,
                                'joseph: very long message very long messagevery long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message  very long messagevery long messagevery long message very long messagevery long message very long messagevery long message very long message very long messagevery long message  very long messagevery long message very long message very long message ry long message very long messagevery long message very long message very long message very long message very long message very long message very long message very long message very long message very long mes'
                              ];
                      
                      Zbxlog::Sender::Send zbx_data={"request":"sender data",
                      "data":[
                      {
                      "host":"Zabbix server",
                      "key":"syslog[]",
                      "value":"joseph: very long message very long messagevery long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message  very long messagevery long messagevery long message very long messagevery long message very long messagevery long message very long message very long messagevery long message  very long messagevery long message very long message very long message ry long message very long messagevery long message very long message very long message very long message very long message very long message very long message very long message very long message very long mes",
                      "timestamp":"1360154656",
                      "source":"user",
                      "severity":"12",
                      "eventid":"13",
                      }
                      ]
                      }
                      Zbxlog::Sender::Send response=ZBXD^AW^@^@^@^@^@^@^@{
                              "response":"success",
                              "info":"Processed 1 Failed 0 Total 1 Seconds spent 0.001075"}
                      It is correctly received by zabbix_server.
                      In zabbix_server.log:
                      Code:
                       27781:20130206:134417.026 Trapper got [{"request":"sender data",
                      "data":[
                      {
                      "host":"Zabbix server",
                      "key":"syslog[]",
                      "value":"joseph: very long message very long messagevery long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message very long messagevery long message very long message very long messagevery long message  very long messagevery long messagevery long message very long messagevery long message very long messagevery long message very long message very long messagevery long message  very long messagevery long message very long message very long message ry long message very long messagevery long message very long message very long message very long message very long message very long message very long message very long message very long message very long mes",
                      "timestamp":"1360154656",
                      "source":"user",
                      "severity":"12",
                      "eventid":"13",
                      }
                      ]
                      }] len 1668
                      and it appears in Zabbix Web frontend with correct facility and severity.

                      You may try to turn on debugging in zbxlog Controller.pm and restart zbxlog. It will dump raw syslog message as received from your device.
                      It will look like:
                      Code:
                      Read:remoteip=127.0.0.1 remote_host=127.0.0.1 buf=<13>Feb  6 13:59:21 zabbix2 joseph: very long message very long messagevery long message very long message very long message very long message very long message very long message
                      It may help find if something specific in your message is breaking zbxlog parser.

                      You may also try to raise $DEBUG to 2 in Sender.pm, you'll get a dump of the full message sent by zbxlog to zabbix_server (as in my logs).

                      Hope this helps,
                      JBo

                      Comment

                      • parcival
                        Member
                        • Sep 2010
                        • 44

                        #161
                        Hi Jbo merci beaucoup,

                        i get this with Debug 2.
                        I do not understand the problem...

                        Code:
                        Read:remoteip=192.168.1.50 remote_host=192.XXX..XXXbuf=<131>fwgate-1: NetScreen device_id=fwgate-1  [Root]system-error-00601: DNS:QUERY:
                        NULL-QUERY has been detected from 91.XXX.XXX.XXX/5976 to 217.XXX.XXX.XXX/53 through policy 114 1 times. (2013-02-06 16:56:19).
                        Zbxlog::Sender::Send item=$VAR1 = [
                                  'fwgate-1',
                                  '',
                                  'syslog[]',
                                  '131',
                                  'local0',
                                  14,
                                  1360166180,
                                  'fwgate-1: NetScreen device_id=fwgate-1  [Root]system-error-00601: DNS:QUERY:NULL-QUERY has been detected from 91.XXX.XXX.XXX/5976
                         to 217.XXX.XXX.XXX/53 through policy 114 1 times. (2013-02-06 16:56:19).'
                                ];
                        
                        Zbxlog::Sender::Send zbx_data={"request":"sender data",
                        "data":[
                        {
                        "host":"fwgate-1",
                        "key":"syslog[]",
                        "value":"fwgate-1: NetScreen device_id=fwgate-1  [Root]system-error-00601: DNS:QUERY:NULL-QUERY has been detected from 91.XXX.XXX.XXX/5976 t
                        o 217.XXX.XXX.XXX/53 through policy 114 1 times. (2013-02-06 16:56:19).",
                        "timestamp":"1360166180",
                        "source":"local0",
                        "severity":"14",
                        "eventid":"131",
                        }
                        ]
                        }
                        Zbxlog::Sender::Send response=OK
                        thx
                        parcival

                        but with windows eventlog will work.
                        Code:
                         "source":"Microsoft-Windows-Security-Auditing",
                                                "severity":8,
                        Last edited by parcival; 06-02-2013, 18:15.

                        Comment

                        • JBo
                          Senior Member
                          • Jan 2011
                          • 310

                          #162
                          I don't understand what is going on, everything looks fine on zbxlog side.
                          I have injected your syslog message by modifying Controller.pm (every received syslog message is replaced by yours):
                          Code:
                              $buf='<131>fwgate-1: NetScreen device_id=fwgate-1  [Root]system-error-00601: DNS:QUERY: NULL-QUERY has been detected from 91.XXX.XXX.XXX/5976 to 217.XXX.XXX.XXX/53 through policy 114 1 times. (2013-02-06 16:56:19).';
                              print STDERR "Read:remoteip=$remote_ip remote_host=$remote_host buf=$buf\n" if $DEBUG>0;
                              my $msg = new Zbxlog::SyslogMsg($remote_host, $remote_ip, $buf);
                              $self->{_queue}->Push($msg) if defined $msg;
                          and I get it correctly in Zabbix (see attached screenshot).
                          Attached Files

                          Comment

                          • parcival
                            Member
                            • Sep 2010
                            • 44

                            #163
                            sigh it's a pity

                            Comment

                            • parcival
                              Member
                              • Sep 2010
                              • 44

                              #164
                              JBo you run with ZBX-2.0.4 ?

                              Comment

                              • JBo
                                Senior Member
                                • Jan 2011
                                • 310

                                #165
                                Originally posted by parcival
                                JBo you run with ZBX-2.0.4 ?
                                zabbix 2.0.4 frontend and server compiled from source:
                                Code:
                                 27073:20130206:132705.169 Starting Zabbix Server. Zabbix 2.0.4 (revision 31984).
                                 27073:20130206:132705.169 ****** Enabled features ******
                                 27073:20130206:132705.169 SNMP monitoring:           YES
                                 27073:20130206:132705.169 IPMI monitoring:           YES
                                 27073:20130206:132705.169 WEB monitoring:            YES
                                 27073:20130206:132705.169 Jabber notifications:       NO
                                 27073:20130206:132705.169 Ez Texting notifications:  YES
                                 27073:20130206:132705.169 ODBC:                       NO
                                 27073:20130206:132705.169 SSH2 support:              YES
                                 27073:20130206:132705.169 IPv6 support:              YES
                                 27073:20130206:132705.169 ******************************

                                Comment

                                Working...