Auto Discovery : why not use arpwatch ?

[onslo]

I built a system some years ago that helped me maintain a list of active ip addresses on my class B network. Initially it did somethng similar to the way Zabbix currently does auto-discovery. I used NMap to scan ranges and detect systems for profiling and adding to the database.

This method was fine until the network grew to such a size that periodic scanning of an entire class b range became really innefficient. I had to find a different method for detecting and profiling hosts.

Enter ArpWatch (http://packages.qa.debian.org/a/arpwatch.html)

Using the arp.dat file generated by arpwatch I was able to detect and begin profiling hosts on my networks without the need for periodic scanning of the entire subnet or range of subnets. This reduced the overhead hugely and allowed me to target devices more directly as they appeared on the network.

I'm a huge fan of Zabbix but I have to say that the current auto-discovery "scanning" mechanism only works for small ip ranges and is not really viable (in my opinion) for enterprise level discovery due to the arp "noise" that scanning in this way creates. It is also cumbersome and requires multiple discovery rulesets to enable class b scanning at an efficient level.

I personally think it would be better to redesign the auto-discovery within Zabbix to adopt arpwatch as it's primary detection mechanism.

I may have a go at integrating it myself using my existing ip monitoring system and will share the results with the community if successful. However, I am interested in others thoughts on the matter.

Rgs

onslo

[Tenzer]

I see one issue with your suggestion though. Arpwatch needs to be running on the same network as the hosts you want to find. The Zabbix server doesn't always to that.

[onslo]

That's a good point. However, there's no reason why arpwatch has to run on the Zabbix server. It could be running on a remote host. All you need access to from the Zabbix server is the arp.dat file.

rgs

Onslo

[Alexei]

I like the idea, however it has one drawback, which seems to be important to me. ZABBIX auto-discovery is based not only on availability of certain IP addresses (devices), but also on availability of services.

I believe arpwatch does not provide any information on services.

Perhaps a combination of existing method with arpwatch would be very efficient.

Also, you may consider taking advantage of ZABBIX Proxies in 1.6, which may perform auto-discovery for remote locations on behalf of ZABBIX Server.

[nelsonab]

I agree, I think it would be good if the Auto Discovery "agent" had the option of polling IP addresses given to it from arpwatch. This would allow for a much more efficient network scan. True Auto Discovery is built to add new services and not just hosts, but you still have to talk to that host/service's IP address. If an IP has never shown up on the network why tie up the time querying if it's there? Now I can see the usefulness of blind scans like now for systems behind a router on a separate network segment and thus the capability should stay.

The current blind scanning approach has a finite limit to it's effectiveness. Adding more proxies or DM nodes to scan will multiply capability but even then it's still inefficient in many cases. Having the ability to poll only hosts seen by arpwatch would improve efficiency and increase the number of hosts that can be scanned per second, less likelihood of TCP/ICMP connect timeouts.

[onslo]

The way i used to handle it was to use arpwatch for the "host detection" and then NMap the detected hosts to find the services.

So, perhaps this is the procedure to follow :

1. Arpwatch detects a host, Zabbix is informed.
2. Zabbix AD rule scans the detected host using user-defined scanning rules.
3. Zabbix actions are applied to the discovered host in the normal way.

I think it should be an option with Zabbix to use a mechanism similar to this instead of the blind scanning. Particularly if you are only interested in one subnet. Blind scanning should still be available but this arpwatch could be the preferred methdo if you can't multihome or proxy your subnets to the Zabbix server.

Rgs

Onslo

[richlv]

would it be possible to integrate arpwatch functionality (either in a new implementation or by importing arpwatch itself) in zabbix server/proxies, so that no new dependency is created ?

[Tenzer]

The smartest way I can see this implemented, is using a (maybe upcoming) API for Zabbix, which can be triggered by some kind of arpwatch-script implementation.

When arpwatch registers a new host, it triggers a script through the -s option (available in Debian/Ubuntu), which allows you to specify which script/program to run when a new host is found. The script should then use the API to let Zabbix know, and Zabbix then figures out what to do from there.

[onslo]

That would work well Tenzer....
At the moment, the way I do it is to read the arp.dat file every "n" minutes, filter out the ip addresses (and mac addresses) and then filter out the ones that aren't already liste din the database, then perform service scans on the results and add the data into the relevant table.

The -s switch implementation would reduce this overhead further as no database or log file polling would be required.

This is starting to sound like a very viable feature !

[onslo]

Looks like there's a distributed version of arpwatch for remote networks that also includes a detection handler for script execution

http://www.crc.net.nz/software/darpwatch.php

[onslo]

Alexei,

Do you think this would be viable for a future release?
If so, when do you think we could expect to see it ?

[onslo]

Ok so 1.6 is released.... any chance of this making the next release ? Alexei ?