Resolve an event with an overwriting log file

[markfree]

Hey guys.
I have this CSV log file that is generated by a script and it is overwritten on every run, meaning it is not incremented. Every time the scheduled script runs, new lines overwrite the data.
The content looks something like this.

20240726 23:09:00;error1;database 1;other;object1;value 1;host1,host2;critical
20240726 23:13:00;error2;database 1;other;object2;value 2;host1,host2;normal
20240726 23:13:01;error2;database 4;backup;object1;value 2;host3;critical
(...)

I've created a simple log item that captures all the file data.

log[/path/to/file.log,"(.*)",,,skip,\1]

With this, a trigger can create an event for each line with a severity of "critical", the 8th column.

find(/host/log[/path/to/file.log,"(.*)",,,skip,\1],1#,"regexp","critical")

Now I need to resolve the event when the same line is added again with a severity of "normal", however many cycles later.


For example:
The following line appears in the log file with a "critical" severity in the 8th column, so it should trigger an event.

20240726 23:09:00;error1;database 1;other;object1;value 1;host1,host2;critical

Ignoring the timestamp, if the same line (with the same data) appears after some cycles with a "normal" severity in the 8th column, the event should be resolved.

20240728 23:20:00;error1;database 1;other;object1;value 1;host1,host2;normal

I'm having trouble resolving the event.
It is not a simple "find()" function as I have to consider the log history.
Any thoughts on this matter?

[markfree]

I've found a contour solution by using global event correlations.

I've written a study scenario for this on GitHub.

[Markku]

Wouldn't it be so that the trigger problem expression is where find() finds "critical", and resolve expression is where find() finds "normal"?

(very quick answer, maybe I missed some crucial detail)

Markku

[markfree]

Thanks for your input, markku.
If in one cycle the file has 3 rows with the critical severity, this generates 3 events since I'm using a log item key.

Let's say that a few cycles later 1 of the 3 rows is set to normal severity. Now, the corresponding event needs to be resolved.

If I were to resolve with just a find function, all 3 events would be resolved, which is not the goal.

[Markku]

Ok, how about some tricks with assigning tags to problems and resolving only problems with matching tags? Now I'm just throwing out ideas that I haven't ever used myself Someone with actual experience of similar cases please chime in.

Markku

[markfree]

I just don't see how right now.
Maybe macro functions can help.

[markfree]

I've found a contour solution by using global event correlations.

I've written a study scenario for this on GitHub.