Ad Widget

Collapse

Monitor DNS blacklist entries

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Arioch
    Junior Member
    • Dec 2009
    • 19

    #1

    Monitor DNS blacklist entries

    I've written a small howto for all those sysadmins who'd like to be notified when their mail servers appear on a DNS blacklist:



    Feel free to post your comments and/or questions over here.

    Happy monitoring.
  • dima_dm
    Senior Member
    • Dec 2009
    • 2697

    #2
    Please attach the sample zabbix-rbl_list.txt.
    Each user should not create a file from zero
    Example:
    http://whatismyipaddress.com/staticp...ss-blacklisted
    Return code (type text):
    Not in RBL: Ok
    In RBL: The list of RBL in which enters a server
    Last edited by dima_dm; 18-02-2010, 09:05.

    Comment

    • Arioch
      Junior Member
      • Dec 2009
      • 19

      #3
      There is a list available as stated in the post:
      Let’s get to it then. First of all we need an up to date list of RBL’s which we can use to check whether we’re listed or not. You could try the list I’m maintaining and using for my own monitoring purposes. The most recent version can be found here. It contains a whopping 93 RBL’s to get you started with.

      Comment

      • dima_dm
        Senior Member
        • Dec 2009
        • 2697

        #4
        Usually use other color of the text for URL.
        RBL list.dsbl.org is closed. See http://dsbl.org/

        If to change a return code
        Not in RBL: Ok
        In RBL: The list of RBL in which enters a server
        RBL List may be included in notification message

        Comment

        • Arioch
          Junior Member
          • Dec 2009
          • 19

          #5
          Alright thanks for pointing that out. Dsbl has been removed from the list.
          I might update the script to include which RBL you're listed in into the notification sometime. However it's not very high on my to-do list right now, priority-wise.

          Comment

          • dima_dm
            Senior Member
            • Dec 2009
            • 2697

            #6
            My version of this script written on perl.
            I use cron script for monitoring Realtime DNS Blacklists (Because maximal timeout for External Check 30 seconds). The program writes data to a temporary file /tmp/zabbix_rbl_status.txt
            On Zabbix Server
            crontab -l -u zabbix
            [email protected]
            0 */4 * * * /etc/zabbix/externalscripts/zabbix_cron_rbl_check.pl

            External Check
            /etc/zabbix/externalscripts/zabbix_rbl_check.pl host_name_or_IP [host_name_or_IP]

            Zabbix template:
            Template_RBL_Check.xml
            Template_RBL_Check_with_IP.xml use macros {$RBLIP}=mx1.hotmail.com to check Realtime DNS Blacklists for mx1.hotmail.com

            RBL List may be included in notification message
            Use in Action Comment field:
            Latest value
            {{HOSTNAME}:{TRIGGER.KEY}.last(0)}


            Example /tmp/zabbix_rbl_status.txt

            100.15.252.133_last_rbl_check: 1267536794 Tue Mar 2 16:33:14 2010
            100.15.252.133_status: 100.15.252.133 in RBL bl.emailbasura.org;
            100.15.252.133_last_check: 1267537814 Tue Mar 2 16:50:14 2010
            mx1.hotmail.com_last_check: 1267537815 Tue Mar 2 16:50:15 2010
            mx1.hotmail.com_status: OK
            mx1.hotmail.com_last_rbl_check: 1267536272 Tue Mar 2 16:24:32 2010
            Attached Files

            Comment

            • aistisz
              Junior Member
              • Nov 2010
              • 1

              #7
              Thanks Dima, works great

              Comment

              • frater
                Senior Member
                • Oct 2010
                • 340

                #8
                I've been using dnsbl checks for a long time now on a lot of 'my' IP's
                I always did this using a script, but when I saw this thread I really wanted to do this in Zabbix.
                It will generate a report file which you can parse with another item (vfs.file.regcount or vfs.file.regmatch) and use that to create triggers.

                It works nice as a package on its own.

                @work we have an IPAUDIT to monitor our traffic. Every day it generates a list with IP's that have been using SMTP heavily. I only need to run my script there without any parameters and I will get a list of all IP's that are listed.

                If this file is not present (probably so, in your case) it will try to obtain your WAN-IP (courtesy of dyndns.org) and use that result.

                Over time I also discovered that those DNSBL-providers suddenly stop. That's why I test the DNSBL-providers themselves before I start querying them.

                I'm a bit too tired now to explain the whole script so I'm just posting it and will come back on it later:

                # cat check_rbl
                Code:
                #!/bin/bash
                #####################################################
                # check_rbl
                #####################################################
                # 17-12-2010 by JP van Melis
                #
                # Use at your own risk!
                #####################################################
                export PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
                
                # If you don't want to use your own DNS-server
                # Use Google's, Blank it if you want to use the default always.
                DNServer=8.8.8.8
                # URL=http://blog.penumbra.be/files/zabbix/rbl_list.txt
                URL=http://wd.mirmana.com/dnsbl.txt
                dnsbl=/etc/dnsbl.txt
                IPAUDITDIR=/home/ipaudit/reports/daily/server2/html
                
                DNSBL=
                TESTDNSBL=0
                LISTIPS=0
                ZABBIX_SENDER=0
                ZABBIX_CONF=/etc/zabbix/zabbix_agentd.conf
                
                TouchedToday=
                Time=2
                DEBUG=0
                
                NAME="$0"
                [ -z "`which readlink`" ] || NAME="`readlink -f "${NAME}"`"
                BASENAME=`basename "${NAME}"`
                reportfile=/var/log/${BASENAME}.report
                
                while getopts zltr:d: name
                do
                  case $name in
                    z)   ZABBIX_SENDER=1;;
                    l)   LISTIPS=1;;
                    t)   TESTDNSBL=1;;
                    r)   reportfile="$OPTARG";;
                    d)   DNSBL="$OPTARG";;
                    ?)   printf "Usage: %s: [-d] [-l] [-t] [-r <reportfile>] [(<IP>|<domain>|<file>)]\n" $0
                    exit 2;;
                  esac
                done
                shift $(($OPTIND - 1))
                
                _exit ()
                {
                  rm -f $ftmp1 2>/dev/null
                  rm -f $ftmp2 2>/dev/null
                  rm -f $ftmp3 2>/dev/null
                  exit $1
                }
                
                if [ ${ZABBIX_SENDER} -eq 1 ] ; then
                  [ -e ${ZABBIX_CONF} ] || ZABBIX_CONF=/opt${ZABCONF}
                  if [ ! -e ${ZABBIX_CONF} ] ; then
                    echo "Unable to find ${ZABBIX_CONF}, can't continue" >&2
                    exit 1
                  fi
                  # zabHOSTNAME=`grep -i ^Hostname ${ZABBIX_CONF} | awk -F= '{print $2}' | awk '{print $1}'`
                fi
                
                ftmp1=`mktemp`
                ftmp2=`mktemp`
                ftmp3=`mktemp`
                
                if [ ${LISTIPS} -eq 0 ] ; then
                
                  # No DNSBL given... use standard file or create one by recursively calling myself
                  [ -z "${DNSBL}" ] && [ -e ${dnsbl} ] && [ ! -d ${dnsbl} ] && DNSBL="${dnsbl}"
                  if [ -z "${DNSBL}" ] ; then
                    echo -e "No ${dnsbl} found...\nI will make an attempt to create one by downloading ${URL}!"
                    if wget -O $ftmp3 ${URL} 2>/dev/null ; then
                      lines1=`cat $ftmp3 | wc -l`
                      # make it look older, so it will get checked next run
                      touch -d "last week" $ftmp3
                      echo "Downloaded ${lines1} DNSBL's, I will now recursively call myself to clean the file up!"
                      $0 -d $ftmp3 -t >${dnsbl}
                      lines2=`cat ${dnsbl} | wc -l`
                      if [ $lines2 -eq $lines1 ] ; then
                        echo "All DNSBL's seem to be valid!"
                      else
                        echo -e "Only $lines2 survived the test!\n\nThese were found to be invalid:\n"
                        while read failed ; do
                          grep -q "${failed}" ${dnsbl} || echo "${failed}" | awk '{print $1}' | sed 's/\.$//' | sed "s/.*/host -tA -W$Time 2.0.0.127.&. ; host -tA -W$Time 1.0.0.127.&./"
                        done <$ftmp3
                      fi
                      echo -e "\nThe default DNSBL list (${dnsbl}) is created, you should not see this anymore"
                      _exit 1
                    else
                      echo "Download from ${URL} failed"
                      _exit 1
                    fi
                  elif  [[ $DNSBL =~ / ]]; then
                    # Create list from file
                    if [ -e "${DNSBL}" ] && [ ! -d "${DNSBL}" ] ; then
                      if head -n1 "${DNSBL}" | grep -q '^[-0-9A-Za-z_.]* [01]$' ; then
                        cp -p "${DNSBL}" $ftmp2
                        TouchedToday="$(find "${DNSBL%/*}" -maxdepth 1 -name "${DNSBL##*/}"  -mtime 0)"
                      else
                        awk '{print $1}' "${DNSBL}" | grep -o '[-0-9A-Za-z_.]*' | sed 's/\.$//' >$ftmp2
                      fi
                    fi
                  else
                    # Create list from command-line parameter
                    echo "${DNSBL}" | grep -o '[-0-9A-Za-z_.]*' | sed 's/\.$//' >$ftmp2
                  fi
                
                  # if file is unchecked or its first line doesn't end with ' 0' oe ' 1'
                  if [ -z "${TouchedToday}" ] ; then
                    # Check if DNSBL-server is working
                    while read blacklistline ; do
                      # remove a trailing dot, so we can add one without making a duplicate
                      blacklist=`echo "${blacklistline}" | awk '{print $1}' | sed 's/\.$//'`
                      if host -tA -W$Time 2.0.0.127.${blacklist}. ${DNServer} | grep -q 'has address 127\.0\.0\.' ; then
                        host -tA -W$Time 1.0.0.127.${blacklist}. ${DNServer} | grep -q 'has address 127\.0\.0\.2' || echo "${blacklist} 1" >>${ftmp3}
                      elif [ ! -z "${DNServer}" ] ; then
                        if host -tA -W$Time 2.0.0.127.${blacklist}. | grep -q 'has address 127\.0\.0\.' ; then
                          host -tA -W$Time 1.0.0.127.${blacklist}. | grep -q 'has address 127\.0\.0\.2' || echo "${blacklist} 0" >>${ftmp3}
                        fi
                      fi
                    done<${ftmp2}
                  else
                    cp -p $ftmp2 $ftmp3
                  fi
                
                  if [ ! -s ${ftmp3} ] ; then
                    echo "No valid DNSBL-servers found"
                    _exit 1
                  elif echo "${DNSBL}" | grep -q '/' ; then
                    if [ -z "${TouchedToday}" ] ; then
                      # If a file is used for the DNSBL's then check if all are valid..
                      # If so, then 'touch it', so it doesn't need checking today.
                      lines1=`cat "${DNSBL}" | wc -l`
                      lines2=`cat "${ftmp3}" | wc -l`
                      [ $lines1 -eq $lines2 ] && touch "${DNSBL}"
                    fi
                  fi
                
                  if [ $TESTDNSBL -eq 1 ] ; then
                    sort -u ${ftmp3}
                    if [ ${ZABBIX_SENDER} -eq 1 ] ; then
                      DNSBLS="`cat ${ftmp3}`"
                      zabbix_sender -c $ZABBIX_CONF -k "dnsbl[dnsbl]" -o "${DNSBLS}" 2>&1 >/dev/null
                    fi
                    _exit 0
                  fi
                
                  [ -s ${ftmp3} ] || exit 1
                fi
                
                # delete surrounding spaces
                IPSTOTEST="`echo "$*" | sed 's/^ *//;s/ *$//'`"
                
                if [ -z "${IPSTOTEST}" ] ; then
                  if [ -d "${IPAUDITDIR}" ] ; then
                    # check IPaudit's daily list
                    fname=`ls -1t ${IPAUDITDIR}/*.html 2>/dev/null | head -n1`
                    [ -e "${fname}" ] && grep -o 'remote_port=25&ip=[0-9.]*' "${fname}" |  awk -F= '{print $3}' | sed 's/^0*//;s/\.0*/./g' >$ftmp1
                  else
                    wget -O - http://checkip.dyndns.org/ 2>/dev/null | grep -io 'IP Address: [0-9.]*' | awk '{print $3}' >$ftmp1
                  fi
                elif echo "${IPSTOTEST}" | grep -q '/' ; then
                  [ -e "${IPSTOTEST}" ] && grep -o '[-0-9A-Za-z_.]*' "${IPSTOTEST}" | sed 's/\.$//' >$ftmp1
                else
                  echo "${IPSTOTEST}" | grep -o '[-0-9A-Za-z_.]*' | sed 's/\.$//' >$ftmp1
                fi
                
                # Convert host names to IP's if it's not done already.
                echo -n '' >$ftmp2
                while read IP ; do
                  if echo "${IP}" | egrep -q '^([0-9]{1,3}\.){3}[0-9]{1,3}$' ; then
                    echo "${IP}" >>$ftmp2
                  else
                    # extract all IP's and concatenate domain to it
                    ResolvedIP="`host -tA "${IP}." | grep -o 'has address [0-9.]*' | awk '{printf "%s ", $3}'`"
                    [ -z "${ResolvedIP}" ] || echo "${ResolvedIP}${IP}." >>$ftmp2
                  fi
                done <$ftmp1
                
                if [ ! -s ${ftmp2} ] ; then
                  echo "NO IP's found" 2>/dev/null
                  _exit 1
                fi
                
                if [ ${LISTIPS} -eq 1 ] ; then
                  cat ${ftmp2}
                
                  if [ ${ZABBIX_SENDER} -eq 1 ] ; then
                    IPS="`cat ${ftmp2}`"
                    zabbix_sender -c $ZABBIX_CONF -k "dnsbl[ips]" -o "${IPS}" 2>&1 >/dev/null
                  fi
                  _exit 0
                fi
                
                echo -n '' >${reportfile}
                while read IPline ; do
                
                  echo -n '' >$ftmp1
                
                  # extract Domain from IPline and cut that Domain out of IPline
                  Domain="`echo "${IPline}" | sed -e 's/[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*//g' | sed 's/ *//g'`"
                  subject="`echo "${Domain}:" | awk '{ printf("%-25s",$0) }'`"
                  [ -z "${Domain}" ] || IPline="`echo "${IPline}" | sed -e "s/${Domain}//"`"
                
                  for IP in ${IPline} ; do
                
                    set ${IP//./ }
                    [ -z "${Domain}" ] && subject="`echo "${IP}:" | awk '{ printf("%-18s\n",$0) }'`"
                
                    while read blacklistline ; do
                
                      blacklist=`echo "${blacklistline}" | awk '{print $1}'`
                      askdns=
                      echo "${blacklistline}" | grep -q ' 1$' && askdns=${DNServer}
                
                      if host -tA -W$Time "$4.$3.$2.$1.${blacklist}." ${askdns} | grep -q 'has address 127\.0\.0\.' ; then
                
                        TXT="`host -t txt -W$Time "$4.$3.$2.$1.${blacklist}." ${askdns} | grep -o 'descriptive text .*' | cut -b19- | sed 's/\"$//'`"
                        echo -n "${subject} ${TXT}" >>$ftmp1
                        [ -z "${TXT}" ] && echo -n "listed @ ${blacklist}" >>$ftmp1
                        echo '' >>$ftmp1
                
                      fi
                    done <$ftmp3
                  done
                  cat $ftmp1 | tee -a ${reportfile}
                done <$ftmp2
                
                if [ -s ${reportfile} ] ; then
                  IPS="`cat ${reportfile}`"
                  zabbix_sender -c $ZABBIX_CONF -k "dnsbl[chk]" -o "${IPS}" 2>&1 >/dev/null
                else
                  echo '.'
                  [ ${ZABBIX_SENDER} -eq 1 ] && zabbix_sender -c $ZABBIX_CONF -k "dnsbl[chk]" -o "." 2>&1 >/dev/null
                fi
                _exit 0
                PS... It seems only 46 of your 96 DNSBL's are actually working (at least according to RFC rules), you may want to check that list you are using.
                Last edited by frater; 20-12-2010, 01:56.
                Zabbix agents on Linux, FreeBSD, Windows, AVM-Fritz!box, DD-WRT and QNAP

                Comment

                • frater
                  Senior Member
                  • Oct 2010
                  • 340

                  #9
                  @penumbra:

                  You may think my script is too complex, but it really does some thorough checking. If you want to keep using the scipt you currently have, you may want to look at this one. I kept most of it to keep it readable.

                  It really checks the result of the DNS-query (127.0.0.2 is what you should get back if you're listed).
                  Code:
                  #!/bin/bash
                  
                  cd /usr/share/zabbix/
                  IP=$1
                  set ${IP//./ }
                  
                  STATUS=0
                  while read i ; do
                      RESULT=$( host -ta -W2 $4.$3.$2.$1.$i. 2>&1 )
                      if echo "$RESULT" | grep -q ' 127\.' ; then
                          #echo .The IP ADDRESS ${1} is listed at $i:\n$RESULT. ## DEBUG
                          let STATUS+= 1
                      fi
                      #echo $RESULT ## DEBUG
                  done < rbl_list.txt
                  
                  echo $STATUS
                  Zabbix agents on Linux, FreeBSD, Windows, AVM-Fritz!box, DD-WRT and QNAP

                  Comment

                  • freefd
                    Junior Member
                    • Feb 2015
                    • 1

                    #10
                    Hello everyone.

                    I rewrote the original Ruby rbl.check (https://github.com/jjmartres/Zabbix/...ipts/rbl.check) script on Perl (https://gist.github.com/anonymous/9b4d5875d3c536e747b6). I would be glad if someone need it.

                    Comment

                    • Edwin Eefting
                      Junior Member
                      • Oct 2015
                      • 4

                      #11
                      Blacklist scanner for huge networks

                      I created a blacklist scanner that scans complete ip blocks.

                      It will create triggers for each IP/blacklist combination.

                      This way you dont have to add all the ip's to zabbix manually.

                      Available at: https://github.com/psy0rz/stuff/tree/master/zabbix-bl

                      Comment

                      • paulgruendel
                        Junior Member
                        • Mar 2016
                        • 6

                        #12
                        @Edwin

                        an how to install your script ?

                        Comment

                        • Edwin Eefting
                          Junior Member
                          • Oct 2015
                          • 4

                          #13
                          Originally posted by paulgruendel
                          @Edwin

                          an how to install your script ?
                          1. import the zbx_pl template into zabbix.
                          2. create a host in zabbix that is linked to this template. call this host "blacklistchecker".
                          3. create a file ips.txt that contain the ip adresses you want to check. you can also use CIDR notation in this file, for example: 1.2.3.0/24
                          4. try running the script manually to see if it works.
                          5. if it works, create a cronjob that will run the script automaticly.

                          more info here: https://github.com/psy0rz/stuff/blob...x-bl/README.md

                          Comment

                          • antipode
                            Junior Member
                            • Jul 2016
                            • 1

                            #14
                            Originally posted by Edwin Eefting
                            1. import the zbx_pl template into zabbix.
                            2. create a host in zabbix that is linked to this template. call this host "blacklistchecker".
                            3. create a file ips.txt that contain the ip adresses you want to check. you can also use CIDR notation in this file, for example: 1.2.3.0/24
                            4. try running the script manually to see if it works.
                            5. if it works, create a cronjob that will run the script automaticly.

                            more info here: https://github.com/psy0rz/stuff/blob...x-bl/README.md
                            Hi Edwin,

                            Thanks a lot for this well packaged work !

                            First I have a technical problem with your script, when launched manually as root :
                            Code:
                            #:./zabbix-bl --zabbix_server 127.0.0.1 --host blacklistchecker --blacklists blacklists.txt --ips ips.txt
                            ...
                            149.xxx.xxx.xxx/32 on virbl.bit.nl: done
                            Sending data to Zabbix server
                            Waiting for low-level-discovery update.
                            Traceback (most recent call last):
                              File "./zabbix-bl", line 163, in <module>
                                if ret[0][0]:
                            IndexError: list index out of range
                            Do you have a clue for resolution ?

                            My env : Zabbix 2.4.8 on Ubuntu 14.04.04 - All files (blacklists.txt ips.txt pyZabbixSender.py zabbix-bl) resides in /usr/lib/externalscripts with good rights. Testing only 8 IP adresses, not an overhelm.

                            Last question, does it support IPv6 adresses ?

                            Hope to hear from you soon.

                            Comment

                            Working...