Query on processing multiple times in a text string against currentness for latency

[Wolfsbane2k]

Evening all.

I've got a custom zabbix item that's monitoring the text output of a process for a number of items, and i'm struggling with my next desired monitoring point, and wondering if anyone might be able to prod me in the right direction?

The text file includes a number of times for items in the system, and i want to process each time in that string to determine how old it is; this can be up to 500 timestamps that i'd like to analyse, and i'm not sure the best way to do it - all i really need to know is the "most latent", and "most recent" time stamp.
I'm thinking it's probably best to create a custom java script item to process the string and report the most and least latent as a dependant item on that source text string, but i'm not sure if there is a better way to do this using fuzzy time or similar?

Has anyone else had a similar issue, and if so, how did you manage to get the data you wanted?

Ta!

[PeterZielony]

What os?

It would be good if you can post some examples and things you got already and example of an output. Its hard to do the guessing

[Wolfsbane2k]

What os?

It would be good if you can post some examples and things you got already and example of an output. Its hard to do the guessing

HI Peter, thanks for that.

We're running Zabbix Server on an Ubuntu 22 system.

The string we're processing is a series of JSON strings from an in-house developed solution, each with a time stamp as an attribute, and we're seeing what we think is caused by latency issues across our custom application that appears randomly, so trying to use zabbix to pick up when symptoms are occurring earlier on in the processing chain and raise alerts to see the issues "live". The number of items in the JSON object varies over time and themselves aren't useful, which is why i'm really trying not to capture each actual value through a LLD discovery to create items for each item, although that would make it easier to just use fuzzytime() against.

I've currently got an dependant item that pulls out all of the time stamps (ISO standard strings) from the JSON into a single array so that I can then iterate through them to run a comparison/sort string against.

Will see if i can extract that list of timestamps as an example.

Edit to add example data of the Jsonpath extract.

["2024-06-26 17:15:01.000", "2024-06-26 16:58:14.000",... ]... etc etc

[PeterZielony]

If i understand correctly

You have application that produces json with dates/times that are fed into zabbix (or taken by agent itself), then you want to take them, sort json with newest on top and calculate time differences between "how long time ago for each of them?

If that's the case it might not be accurate as each calculation will take time and won't be accurate. What margins of errors can you accept? in milliseconds, seconds or minutes?

Is the file you observe rotating or a generic file? or do you use a trapper to get values from the system ? Also on what OS file resides?




I'm trying to understand the process.. is single item can be somehow tied to further processing in system to detect time difference between point A and point B

For example
Item with json produced with timestamp has been generated and left Point A
Item produced arrived at Point B with timestamp

something to correlate those actions - this will be the best to calculate time difference. You could have 2x separate items for each "json" that can extract times and 3rd item that will calculate time difference between them .. but this might be a bit complex but more accurate

Fuzzytime is good sometimes but bear in mind there are delays in calculation for each time due to processing cycle


What i don't understand it chain of actions and purpose of produced json and what it could represent as a "delay"

[Wolfsbane2k]

I

Thanks for that, and apologises for delay in responding!
It's basically a heartbeat timestamp of each app running a cycle being written to a file, which is then picked up by a different app and consolidated into a single JSON text file which includes a number of parameters alongside the timestamp. I'd wanted to process the latency within the intermediate app and reporting the delta as a new parameter, but that's opened an entire can of worms, so trying to workaround it using zabbix.

I'm generally looking for something in the region of 3 minutes since last update; we're sampling everything every 1 minute.

I'm going to have another experiment with JS as i've been offered a "tame" JS developer for a few days!