alternative for syslogs

[da_oli]

Hi all,

i'm not sure if i'm right in this forum, but i'm searching for an equivalent for a syslog server.
we have ariund 150 linux boxes and we tried to use syslog.
But its not possible to search inside because the mysql grows to fast.

But i think zabbix isn't an alternative for this kind of data collection.

What are you using in your environment for collecting log files?

Greets

Oliver

[JBo]

Hi,

Zbxlog may help you.
You can check this thread for a discussion about it.

Regards,
JBo

[eskytthe]

Hi,
I think syslog is ok as "transport" service for central log management. But we need a ”clever” tool to analyses or filter the logs.

OSSEC should be good for security…
SPLUNK is more general propose tool I think – dumps your log into a database and make index on them for easy search. Plugins for different needs (firewall logs etc.) Free if under 5 GB log pr. day, as I remember.
(http://forums.anandtech.com/archive/...t-2066691.html)

I plan to make this outside zabbix and just let a central log server forward interesting or critical log entries to the zabbix server.
BR
Erik

[zabbix_zen]

Splunk has a great interface and greatly eases log analyzing.

Problem is its free version only processes 500Mb/day,
quite slim for anything bigger than a multi-app test scenario
and its pricing is a little unforgiving on scenarios with a large log volume ..

[ghoz]

I just saw a video about something called logstash,
suposedly good for log /events massaging/searching (and opensource)

To stay on topic, it can interact with nagios, could be interesting to see how it could be used with zabbix ...

[Colttt]

in rsyslog you can write all logs in an SQL-Database.. for example mysql and for reporting you can use loganalyzer.. in Debian it gives an extra package rsyslog-mysql

[da_oli]

we use rsyslog with mysql
but if you try loganalyzer within a 40+gb database you wait till ever

we split up in seperate syslog tables and delete stuff older then 30 days, so we are able to search too instead of grabbing data only

[zabbix_zen]

We're considering adopting logstash,
it already allows outputting to Zabbix through zabbix_sender

[lukemacneil]

I'm just using zabbix, on 7400 hosts.. I monitor about 5 logs, messages, daemon, mysql, and a couple custom ones.

Thing is, instead of bringing up the whole log, I only bring up the lines I care about using custom regexes. 1 regex for each log I monitor. like so:

Administration-General-Regular Expressions.
Name r_Messages
Expression: error,failed,warning,CHECK CONDITION,resync,abort
Expression type: Any character string included
Delimiter ,

Then, in my log[] item:
log[/var/log/messages,@r_Messages,20]


If I need to add a string, I just add it to the regex.

In my triggers, I use regexp like so:
{Template_Production:log[/cvsrx/cvsDB/mysqld.err,@r_Mysql,20].regexp(".*InnoDB: Error.*",#20)}=1

[vlturner]

INN: It's not just for porn...

INN does't get the respect it should have as a messaging bus.

http://www.isc.org/software/inn Same folks who bring you BIND DNS

It is very easy to plumb syslog to INN. So that you can then have the syslog messages treated as message feeds. This way you can ship them wherever you need them. I was working on plumbing the feeds into Zabbix so that I can have systems worldwide post messages, and then have the grouping set up, such that consumer processes can receive them for processing.

This way I can have zabbix tracking message feeds for metrics, and have zabbix send alerts as INN messages. Now when they get jabber integrating NNTP this will help in the uniformity of the messages in XML format.