Log File Trigger With Latest Beta

[matt_voss]

Although the item log[XXX] is working fine and storing info in the history_log database table - I am wondering how to setup a trigger for the log file.

I tried using the str(XXX) but that did not appear to work - can anyone give me an example of a valid trigger on a log file?

FYI I've tried syntax such as this:

{hostnameX:log[/var/log/rsync.log].str(writefd)}=1
{hostnameX:log[/var/log/rsync.log].str(writefd)}=0

Thanks!

[Alexei]

No trigger functions are supported for log files yet. It will be introduced in 1.1beta5.

[matt_voss]

Any additional docs to get log files working now in beta5?

BETA5 accepts the trigger but it appears to take forever reading the log file [I.E. I don't know how long it will take (days?) to read the long log file and go to the end where I care about].

It would be more ideal to do a tail -f type of thing then read in the entire log file - is that part of the feature set but I'm not just aware of how to use it yet?

Thanks!

[Alexei]

When you start monitoring a log file, ZABBIX agent sends the whole file to the server and then basically does tail -f.

[Wolfgang]

When you start monitoring a log file, ZABBIX agent sends the whole file to the server and then basically does tail -f.

Given a log filesize of about 500kByte and an intervall of 60sec per host.
Would that mean that each client sends every 60sec 500kByte logfile to the sever?

If this is the case:
- I suspect (without having tried this), that it will break for example low-bandwith vpn/wan connections.
- An idea to solve this would be to store the filepointer of the last read on the client. This way only the diffs to the prevoius check would be transmitted.
However, one would have to manage logrotation. For example i use amavis-stats which works this way. If the filesize of the logfile is smaller than on the previous check, amavis-stats looks for the archived rotated file instead.

[Alexei]

In fact the interval does not affect anything for log file's items. Not that:

1. ZABBIX agent does not send more than 5 lines of log file per second. This is kind of protection from overloading.
2. Last check (file size) is stored on the server
3. Log rotation is supported

[Wolfgang]

Thank you. New question came up

1. ZABBIX agent does not send more than 5 lines of log file per second. This is kind of protection from overloading.

I just did a grep -c 1 /var/log/mail.info.0 on my linux mailserver. It returned
13939 lines.
Would that mean it would take 13939 / 5 = 2787 seconds to get it transfered to the zabbix-server?

And how would it scale if i would monitor also the auth.log or any other logfile like xferreport.log, even on more hosts?

Thanks Wolfgang

[Alexei]

I just did a grep -c 1 /var/log/mail.info.0 on my linux mailserver. It returned
13939 lines.
Would that mean it would take 13939 / 5 = 2787 seconds to get it transfered to the zabbix-server?

And how would it scale if i would monitor also the auth.log or any other logfile like xferreport.log, even on more hosts?

Actually ZABBIX transfers no more than 10 lines per second, not 5. It doesn't change much though.

Yes, it would take 2787/2 seconds to get it transferred. The limit is a result of currently used approach: send one line per each TCP connection. As soon as we completely move to a new XML-based communication protocol the limit will be eliminated, i.e. raised to tens of KB per second.

[Wolfgang]

...As soon as we completely move to a new XML-based communication protocol the limit will be eliminated, i.e. raised to tens of KB per second.

I see. However even then, it would require the bandwith and might run in congestions like in the example given for a WAN or VPN connect.

How do you think about the approach to do the "sort of tail" on the client side.
This way, the traffic over the net would be certainly less.

However, to store the last filepos on the client would allow to cheat the NMS system. Not so good.

A way around this would be to store the filepos of the last check on the server and pass it back to the client on the next request. This in turn might require an entire different logic though...

What do you think?

[Alexei]

How do you think about the approach to do the "sort of tail" on the client side.
This way, the traffic over the net would be certainly less.

It is already implemented as the "sort of tail" on the client side. The agent connects to the ZABBIX server on startup, gets the latest positions and then works in standalone mode. There are no overhead in communication protocol.

[Wolfgang]

My appologise

I think i really got you wrong when i read this.

When you start monitoring a log file, ZABBIX agent sends the whole file to the server and then basically does tail -f.

I understood that the the client would pass the entire log _every_ time to the server and then the server would perform a sort of tail on the received data.

It is already implemented as the "sort of tail" on the client side. The agent connects to the ZABBIX server on startup, gets the latest positions and then works in standalone mode. There are no overhead in communication protocol.

Now it seems to me, that the client passes the entire only once to the server. After that, only updates of the logfile are passed from the client to the server.

So the initial transfer might take it time - but only one first time.
Is this correct?

[Alexei]

So the initial transfer might take it time - but only one first time.
Is this correct?

Absolutely!