Tables and dynamic data

[Frozen]

Hi,

Looking at Zabbix for a major institution, however I see that it needs a few new features before we can use it. I'm interested in developing those new features but I would like to know how you core zabbix developers what this type of development.

What I'm looking at are dynamic tables, lots of snmp and other statistics too are table based and change dynamically. Look at the process table, or file system mount table, network interface tabel etc..

We need a way to gather data and select data with in that set dynamically.

Like this:

Network interface table -> consist of one or more rows of: interface name, mac address, ip address, packages sent, packages recived, mtu .... and so forth and so on..

What we need is to be able to spec, the table and then collect all the data that that table has furthermore if the number of rows goes up or down we need to either stop or start collecting data for that row. Like if we add a new disk on a client well then we automatically need to collect data for that disk.

I envision this as snmp to begin with, that we give a table OID and that we collect all the data for that table. That we later on select what rows to graph or alert on.

The question to the zabbix devloper is how they want to spec a table, how do they want to collect the data, and how do they want to feed that data into the DB? I can go a head and do this all by my self but I rather want to know how you want this done.. i.e. the flow of data and how you want it arranged in the DB..

Cheers O

[Alexei]

Hi,

Looking at Zabbix for a major institution, however I see that it needs a few new features before we can use it. I'm interested in developing those new features but I would like to know how you core zabbix developers what this type of development.

What I'm looking at are dynamic tables, lots of snmp and other statistics too are table based and change dynamically. Look at the process table, or file system mount table, network interface tabel etc..

We need a way to gather data and select data with in that set dynamically.

Like this:

Network interface table -> consist of one or more rows of: interface name, mac address, ip address, packages sent, packages recived, mtu .... and so forth and so on..

What we need is to be able to spec, the table and then collect all the data that that table has furthermore if the number of rows goes up or down we need to either stop or start collecting data for that row. Like if we add a new disk on a client well then we automatically need to collect data for that disk.

I envision this as snmp to begin with, that we give a table OID and that we collect all the data for that table. That we later on select what rows to graph or alert on.

The question to the zabbix devloper is how they want to spec a table, how do they want to collect the data, and how do they want to feed that data into the DB? I can go a head and do this all by my self but I rather want to know how you want this done.. i.e. the flow of data and how you want it arranged in the DB..

Cheers O

I am not sure what you are going to do with, for example, the process table. The table will change frequently. Will you collect data from all processes? Is it really required? The amount of data and overhead would be huge for relatively small (hundreds) number of servers.

I think that the feature (browsing of available tables and data) will be extremely useful when it comes to manual management of items and auto-discovery, however I am not sure Zabbix shall start monitoring of something just because the something (process, interface, filesystem, etc) is available.

Zabbix follows "monitor what you need" philosophy rather than "collect everything and be buried" approach.

[Frozen]

I am not sure what you are going to do with, for example, the process table. The table will change frequently. Will you collect data from all processes? Is it really required? The amount of data and overhead would be huge for relatively small (hundreds) number of servers.

I think that the feature (browsing of available tables and data) will be extremely useful when it comes to manual management of items and auto-discovery, however I am not sure Zabbix shall start monitoring of something just because the something (process, interface, filesystem, etc) is available.

Zabbix follows "monitor what you need" philosophy rather than "collect everything and be buried" approach.

You are quite right about that. I was envisioning that, you would get the data say the filesystem table and then select which once you want to collect data for and maybe have have alarms for.

The same can be said for e.g. the process tabel maybe you are monitoring a specific process and how much cpu or memory that process is using.

In regards to file system at the moment the client is monitoring / and /usr separately even if they are the same thing. I want to avoid this, dynamic data will enable us to avoid it..

Cheers O

PS: Sorry got to rush.. more later

[mcarbonneaux]

i've the same need... to finaly migrate all my cacti to zabbix...

no the process table is very large and very moving...is interesting to monitor one process name and the agregated memory of this process... but is more usefull for file system (like say frozen) ... for disk...
and for me for network interface...

for example, with nortel (other switch constructor do also that) switch that have port slot card...
you can add switch port by adding card...
each port are seen as network interface...
and port index of each of thes interface in iftables are dependante on the possition of the slot of the added card...

to monitor this type of switch with zabbix you need know how many card and position of this card where is.
to do template for this type of host you need one template for each card position (with an item for error,in,out packet for each port of the slot)...
and link each template for each slot installed on the host to monitor it...
and to make each template i'm oblige to add all item (24 port par slot) manualy beacause each port have positional index... ok i've used import/export xml to accelarate this...

but in cacti i need only to define snmp query xml file to get all information of all interfaces for each metric i need for each port in iftables without need to know index position of each interface and how many interface that i have on each host...

if i add card on slot, or install new switch with 2 or 8 slot, i need only to configure only at one time the switch zabbix with that possibilty, at the first installation or on autodiscovry... and i can make only one template for all my nortel switch (i've near 300 switch to monitor many differtent configuration)...

http://www.cacti.net/downloads/docs/...query_xml.html
http://docs.cacti.net/?q=node/301

another example (from http://www.roth.lu/cacti-snmp-tut.htm):

<interface>
<name>Get SNMP IPs</name>
<description>Queries a host for a list of IPs with traffic monitored by iptables</description>
<oid_index>.1.3.6.1.4.1.2021.49.42.101</oid_index>
<oid_num_indexes>.1.3.6.1.4.1.2021.49.42.100.1</oid_num_indexes>
<index_order>ipIP:ipIndex</index_order>
<index_order_type>numeric</index_order_type>
<index_title_format>|chosen_order_field|</index_title_format>

<fields>
<ipIndex>
<name>Index</name>
<method>walk</method>
<source>value</source>
<direction>input</direction>
<oid>.1.3.6.1.4.1.2021.49.42.101</oid>
</ipIndex>
<ipIP>
<name>IP Address</name>
<method>walk</method>
<source>value</source>
<direction>input</direction>
<oid>.1.3.6.1.4.1.2021.49.101</oid>
</ipIP>
<ipInBytes>
<name>Incoming Traffic</name>
<method>walk</method>
<source>value</source>
<direction>output</direction>
<oid>.1.3.6.1.4.1.2021.50.101</oid>
</ipInBytes>
<ipOutBytes>
<name>Outgoing Traffic</name>
<method>walk</method>
<source>value</source>
<direction>output</direction>
<oid>.1.3.6.1.4.1.2021.51.101</oid>
</ipOutBytes>
</fields>
</interface>

the idea is to have the possibility to create item to get complete or partial table value.

[richlv]

The question to the zabbix devloper is how they want to spec a table, how do they want to collect the data, and how do they want to feed that data into the DB? I can go a head and do this all by my self but I rather want to know how you want this done.. i.e. the flow of data and how you want it arranged in the DB..

Cheers O

interesting, especially db part. which process should update the db ?
for this to be of any use, zabbix_server, i guess - otherwise no timely reaction to changes seems possible.