Problems with regex preprocessing

[cyberphantom]

Hello
I'm configuring some items with regex preprocessing, to return some specific results. The problem is that my master item returns a big response, which is transformed into LLD macros, here's a example:

[{"{#X}":"abc","{#Y}":"509","{#Z}":"SA"},{"{#X}" :"d ef","{#Y}":"577","{#Z}":"SE"},{"{#X}":"ghi","{#Y }" :"502","{#Z}":"SI"},{"{#X}":"jkl","{#Y}":"540 "," {#Z}":"SO"}(...) etc etc...
I generate many dependents items of this master with discovery rule. My objective is that each item returns their (#Y) value, respectively. To do this, each generated item has a preprocessing with regex to be generated with {#Y} value.
My regex was: {#Y}":"(.*?)"
It worked (at least on regex101) but my items become disabled because zabbix interpret {#Y} as a LLD macro in the output, and the return is something like: 509":"(.?)".
I've tried Y}":"(.*?)" and it works, but regex just get the first match and all the other items get the same result of the first item, and this is not true...
Help!

[ISiroshtan]

OK, so do the following:

1. Modify your script to return data with no macro-like syntax. For example I modified it to:

[{"Name":"abc","Value":"509","Location":"SA"},{"Name":"def","Value":"577","Location":"SE"},{"Name":"ghi","Value":"502","Location":"SI"},{"Name":"jkl","Value":"540","Location":"SO"}]

2. In discovery in Macros define 1 macro like

LLD Macro: {#NAME}
JSONPath: $.Name
(I use Name in $.Name so it corresponds with modified data, if you used different field in modified data for ex-{#X} - use that one instead)

3. In Item prototype define 2 pre-processing rules:

1st

Name: JsonPath
Parameters: $[?(@.Name == "{#NAME}")].Value
(if defined LLD Macro name not as {#NAME} but as smthing different - change it here to that)
(if you changed ex-{#X} into something other then Name - use modified param name here in place of @.Name)
(if you changed ex-{#Y} into something other then Value - use modified param name here in place of )].Value)

2nd

Name: Trim
Parameters: ["]

4. Run and test

Ping me up if something not works for you so we can try look.

[ISiroshtan]

Could you elaborate what you want get in result on item level? Do you need concat of all {#X} for example? Like for {#X} item it should be "abcd ef ghi jklHD"?

Also Zabbix regex engine does not use GLOBAL parameter so you need to disable it on regex101 (same goes for multiline option) (on regex101 site on the right right of expression field there is /gm - click it and uncheck both options)

[ISiroshtan]

Also, as far I can say, your input is JSON? Then why do you use regex and not JSONPath? Doing

$[*].["{#X}"]

will result

["abc","d ef","ghi","jklHD"]

Is it what you want to get in the end?

[cyberphantom]

Thanks for your reply!
Hmm... what I want is this:

each item value with their respective values. Like the first one with "509" result, the second one with "577" etc... as I showed in my regex example above
And yes, these items were generated by a master item type "database monitor" and with the key "db.odbc.discovery", so this key transform the output in a JSON array for LLD Macros, that's why my output is like this

[{"{#X}":"abc","{#Y}":"509","{#Z}":"SA"},{"{#X}" :"def","{#Y}":"577","{#Z}":"SE"},{"{#X}":"ghi", "{# Y }" :"502","{#Z}":"SI"},{"{#X}":"jkl","{#Y}":"540 "," {#Z}":"SO"}(...)

I use regex cause I'm more confident with this..

[cyberphantom]

Could you elaborate what you want get in result on item level? Do you need concat of all {#X} for example? Like for {#X} item it should be "abcd ef ghi jklHD"?

Also Zabbix regex engine does not use GLOBAL parameter so you need to disable it on regex101 (same goes for multiline option) (on regex101 site on the right right of expression field there is /gm - click it and uncheck both options)

I've done what you recommend on regex101 and yes, the regex is matching just with the first value... sad
Tried to use JSONPATH but I received, for my first item:

1. Failed: cannot extract value from json by path "$[*].["504"]": no data matches the specified path

I used this JSON> $..[{#Y}]

don't know what to do...

[ISiroshtan]

So to clarify, your master item is actually returning {#X} inside a value?

And is it by chance some custom script that you can modify to replace MACRO like syntax into some more manageable values?

[cyberphantom]

The master item is returning: [{"{#X}":"abc","{#Y}":"509","{#Z}":"SA"},{"{#X}" :"def","{#Y}":"577","{#Z}":"SE"},{"{#X}":"ghi", "{# Y }" :"502","{#Z}":"SI"},{"{#X}":"jkl","{#Y}":"540 "," {#Z}":"SO"}(...)] , so the value that I want is just the {#Y} values, like "509", "577", "502"... as the values of my items
hmmm yes, I can modify this, but I really need the macros

[ISiroshtan]

So I made a small script that roughly emulates your environment (just a .sh script that echoes your modified sample data back to Zabbix). Result:




Please advise is it what you wanted to achieve.

[cyberphantom]

So I made a small script that roughly emulates your environment (just a .sh script that echoes your modified sample data back to Zabbix). Result:




Please advise is it what you wanted to achieve.

That's it!!

[ISiroshtan]

OK, so do the following:

1. Modify your script to return data with no macro-like syntax. For example I modified it to:

[{"Name":"abc","Value":"509","Location":"SA"},{"Name":"def","Value":"577","Location":"SE"},{"Name":"ghi","Value":"502","Location":"SI"},{"Name":"jkl","Value":"540","Location":"SO"}]

2. In discovery in Macros define 1 macro like

LLD Macro: {#NAME}
JSONPath: $.Name
(I use Name in $.Name so it corresponds with modified data, if you used different field in modified data for ex-{#X} - use that one instead)

3. In Item prototype define 2 pre-processing rules:

1st

Name: JsonPath
Parameters: $[?(@.Name == "{#NAME}")].Value
(if defined LLD Macro name not as {#NAME} but as smthing different - change it here to that)
(if you changed ex-{#X} into something other then Name - use modified param name here in place of @.Name)
(if you changed ex-{#Y} into something other then Value - use modified param name here in place of )].Value)

2nd

Name: Trim
Parameters: ["]

4. Run and test

Ping me up if something not works for you so we can try look.

[cyberphantom]

Tested! Items aren't returning nothing now but are all enabled
When test the preprocessing on each item:
"cannot extract value from json by path "$[?(@.Name== "abc")].Value"

I've changed all values according to my environment, and the return is like this according to your example code, representing mine

[ISiroshtan]

Show me something to work with?

Like your modified data, preprocessing rule on new items that were discovered

[ISiroshtan]

Also, you did modify the script used for discovery AND the script that actually sends data to master item? (if it's different scripts)

[cyberphantom]

I've changed just the key that transform it to LLD macros (cause in database monitor, we have this 2 options), now the output of both are just in JSON format, like this:
[{"Name":"abc","Value":"123","Local":"xxxxx","Total ":"AAA","Name":"abc","Value":"456","Local":"xxxxx" ,"Total":"AAA","Name":"abc","Value":"789","Local": "xxxxx","Total":"AAA" etc...}]
so now the output is like this, without LLD macros

[cyberphantom]

Okay, it seems that it worked for some items.... lol
There's some items that are with the values correctly! I'll see if it's just a slowness