Hello everyone, I am wondering whether it is possible to make Zabbix send an alert or start an action when some percentage of items or triggers are active.
To explain the background: We have a program running on our server which has several active threads running at once. To keep the math simple for a proof-of-concept, let's say that there are 10 threads running at once. We know that a thread is active because each thread generates log entries with its thread number in a central log file. So, for example, you might look in the log file and see something like this:
Thread 8 active
Thread 4 active
Thread 9 active
...and so on. The order is random, but each thread should pop up at least every 60 seconds, or else the thread has probably stalled somewhere.
What we'd like to do is trigger an alert when some percentage (let's say 20% for example) of these threads are no longer running. We'll know that a thread has fallen off the map if it doesn't show up in the log file for 60 seconds, but we don't necessarily want to trigger an alert if just one thread drops out, as we can tolerate the loss of a single thread. We'd only like to generate an alert if multiple threads drop out simultaneously.
It's easy to make an item that looks for the text "Thread 1 active" and then trigger if that text doesn't show up in the log file for 60 seconds, but how can we create a trigger or an alert which only activates when some percentage of these triggers are active at the same time?
To explain the background: We have a program running on our server which has several active threads running at once. To keep the math simple for a proof-of-concept, let's say that there are 10 threads running at once. We know that a thread is active because each thread generates log entries with its thread number in a central log file. So, for example, you might look in the log file and see something like this:
Thread 8 active
Thread 4 active
Thread 9 active
...and so on. The order is random, but each thread should pop up at least every 60 seconds, or else the thread has probably stalled somewhere.
What we'd like to do is trigger an alert when some percentage (let's say 20% for example) of these threads are no longer running. We'll know that a thread has fallen off the map if it doesn't show up in the log file for 60 seconds, but we don't necessarily want to trigger an alert if just one thread drops out, as we can tolerate the loss of a single thread. We'd only like to generate an alert if multiple threads drop out simultaneously.
It's easy to make an item that looks for the text "Thread 1 active" and then trigger if that text doesn't show up in the log file for 60 seconds, but how can we create a trigger or an alert which only activates when some percentage of these triggers are active at the same time?
just a wild idea...
Comment