AI for Zabbix 1.1

[pabloadmin]

Hi Alexei,

We have a big idea, but we don't know if it posseble add to zabbix 1.1. The idea is the follow:
Zabbix can send mails and is possible send mails with xml content, we think that is possible
to connect zabbix with some AI program (Artificial Intelligence). Some AI programs can recive mails
with xml content and intepreted it, if the mail say service is down for 5 minutes (for example) the AI
can start it. The Ai can take inoffensive actions something like:

- Start a service down
- Delete tmp files, if you have lower disk space
- Discover a new device connected to any switch and add this host to zabbix.
- Kill any secundary process

Many things, we saw the Alice open source project at http://www.alicebot.org
If is possible, can be a great idea to zabbix 1.1!!!
You can add more sophisticated functions to zabbix.

thank you in advance, we wait your answer...
:)

[Alexei]

We have a big idea, but we don't know if it posseble add to zabbix 1.1. The idea is the follow:
Zabbix can send mails and is possible send mails with xml content, we think that is possible
to connect zabbix with some AI program (Artificial Intelligence). Some AI programs can recive mails
with xml content and intepreted it, if the mail say service is down for 5 minutes (for example) the AI
can start it. The Ai can take inoffensive actions something like:

- Start a service down
- Delete tmp files, if you have lower disk space
- Discover a new device connected to any switch and add this host to zabbix.
- Kill any secundary process
I think that ZABBIX can perform all this (some pretty obvious scripting required) with built-in intelligence without help of a third-party brain :)

Many things, we saw the Alice open source project at http://www.alicebot.org
Yes, I know about this project, however I do not see how Alice could help ZABBIX. I'd be very happy to discover Alice is able to write C and(or) PHP code, this would save a lot of my time!

If is possible, can be a great idea to zabbix 1.1!!!
You can add more sophisticated functions to zabbix.

thank you in advance, we wait your answer...
:)

On a serious note, I was thinking about adding neural network functionality to ZABBIX core (not in 1.1 of course). This would help to discover anomalies in behaviour of servers.

Imagine a database server which works 24x7x365. ZABBIX could learn disk I/O, CPU usage, memory I/O, network I/O average pattern, and the AI would help to discover various strange events and notify database administrators. ZABBIX could generate messages like "Hey, I think something wrong with disk I/O. It is normally about 1MB/s, now it is below 0.2MB/s. What's up?"

I have no idea if this really will be useful, and also I do not know how this affect requirements for CPU resources. More evaluation work and reading is required.

[pabloadmin]

Yes, Ai is very complex but maybe alice and zabbix can comunicate by mail, this is easy. Then alice can interpret the message and take a desicion, for example, if proc_count < 1 alice can be connected utilizing (for example) ssh to unix host and start it by script.
This isn't the eighth wonder but is a beginning...


thank you for your time.

[limo]

AI is very good idea. I am not sure if it is needed to implement directly into zabbix or connecto throw xml. I think second variant is better, because Alex can sleep more time, not to work :) and every software will do his work very good :)

On our university, we started to develop such project to "predict" some unusual activities on network. We do not think that AI should have rights to do somethink another than send message to admin. I would not see my servers doing something without my intervence :)

But it is really very good idea to use AI !

[pabloadmin]

Yes, i think that... Ai can be a external process for zabbix then they can comunicate by xml email. Thank you LIMO for your time...

[antifuchs]

Imagine a database server which works 24x7x365. ZABBIX could learn disk I/O, CPU usage, memory I/O, network I/O average pattern, and the AI would help to discover various strange events and notify database administrators. ZABBIX could generate messages like "Hey, I think something wrong with disk I/O. It is normally about 1MB/s, now it is below 0.2MB/s. What's up?"

I have no idea if this really will be useful, and also I do not know how this affect requirements for CPU resources. More evaluation work and reading is required.

I've been thinking about checking for things like significant changes for a long time, and I think they can indeed be very useful. The disk I/O example above wouldn't even require much of a brain - it's "just" simple statistics:

say this hour's (or any delta you want) average is avg_t, the previous hour's average is avg_y, and this standard error and the previous standard error are stderr_t and stderr_y, it should trigger if: abs(avg_t - avg_y) > 3 * sqrt(stderr_t^2 + stderr_y^2). For some volatile checks it might be good to have another requirement that the difference in averages is greater than 5% of both averages.

CPU requirements for many of these checks could be pretty high, but caching the last average & standard error should be possible. Another thing to consider is if you can have the MySQL server do the (average & standard error computation) work for you. It's what I do in my SBCL benchmark graphs (http://sbcl.boinkor.net/bench/).

A rate-of-change based test would be pretty cool, too. Things like "number of dialup users dropped by more than 400 in the last 10 minutes" or "temperature rose by 3 degrees in the last 1 hour" would probably make quick diagnoses of problems much easier... bonus points for percent-based checks like "temperature rose by 10% in the last hour" (-:

[Alexei]

A rate-of-change based test would be pretty cool, too. Things like "number of dialup users dropped by more than 400 in the last 10 minutes" or "temperature rose by 3 degrees in the last 1 hour" would probably make quick diagnoses of problems much easier... bonus points for percent-based checks like "temperature rose by 10% in the last hour" (-:
This is already supported! You may calculate something like "last()-max(600)" to get a trigger "Number of dialup users dropped by more than 400 in the last 10 minutes".

Thanks for your input!

[antifuchs]

This is already supported! You may calculate something like "last()-max(600)" to get a trigger "Number of dialup users dropped by more than 400 in the last 10 minutes".


That's great! I hadn't spotted that one when I read the documentation. Thanks!

[pabloadmin]

thanks alexei, day by day zabbix is more flexible.
i want to test this new functions, where can i find it?
Is it on Alpha 7?

[Alexei]

thanks alexei, day by day zabbix is more flexible.
i want to test this new functions, where can i find it?
Is it on Alpha 7?What functions you're asking for?

[pabloadmin]

I marked the functions in red color:

abs(avg_t - avg_y) > 3 * sqrt(stderr_t^2 + stderr_y^2).

They work at Zabbix 1.1 Alpha 7?


thank you.

[Alexei]

I marked the functions in red color:

abs(avg_t - avg_y) > 3 * sqrt(stderr_t^2 + stderr_y^2).

They work at Zabbix 1.1 Alpha 7?


thank you. No, they don't. Actually I have no idea when such functions could be included into ZABBIX. It all depends on demand and priorities.

[pabloadmin]

excuse me, I had understood that already they were included.