Hello
I use Zabbix version 4.2.6 in RHEL 7. According to the documentation, this version should be able to preprocess item results using JSONPath. Nevertheless, in practice it is impossible to use it, except from implementing very basic filters.
Per instance, I created an item that receives the following JSON data:
[ {"result":"1", "charged":11111.00, "reported":22222.00, ...more fields here...}, ...more objects here... ]
I set the item to display 'Text' during the debugging phase, to avoid any conversion issues.
Now, assume that I want to extract the value of the field 'reported' when result==1. I did the following attempts on the way to find the magic filter for that:
Attempt 1:
JSON Path expr: $.[?(@.result=="1"]
Result: WORKS! It returns the object corresponding to result==1. How nice
Attempt 2:
JSON Path expr: $.[?(@.result=="1")].reported
Result: WORKS at first (returns [22222.00]) BUT it can fail later (see below)
Attempt 3:
JSON Path expr: $.[?(@.result=="1")].reported.first()
Result: FAILS! No error recorded in the logs.
Error message: Preprocessing failed for: [ {"result":"1", "charged":11111.00, "reported":22222.00, ...}, ... ]
I then removed the '.first()' part to return to the previous working version, but to my amazement, the item continued to fail with the same type of error. No matter how much I waited the item never recovered.
I cloned the item to create a new one - just changing the name & key - but I still got the same error! It seems as if the item, once got marked as failing, it will be considered failing for the rest of its life; similarly for its cloned selves.
Attempt 4:
I started again with a fresh new item. No problems this time! I'm back to the success I had on Attempt 2. Then I went on and added an extra action just for kicks (only to check the behavior, as the anticipated result is not yet meaningfully parsed). I added a simple trimming action:
JSON Path: $.[?(@.result=="1")].reported + Trim: []"
Result: FAILS! Same old story. Same even if I use Regex "\[(.*)\]" or any other random action. Zabbix does not allow me to workaround the issue by using extra steps!
Now, the very interesting bits, are that:
1. All JSON Path filters used are tested within the Zabbix item (in the preprocessing tab) and are SUCCESSFUL. Even third party testers prove the filters are OK.
2. Even in the failing cases, the 'Latest Data' page displays the CORRECT value! So indeed Zabbix went through all steps and created the expected result. Nevertheless, the row is marked with the above error and the item is never graphed.
3. When I add extra preprocessing steps - causing the preprocessing to fail - the error appears on the first step, i.e. the JSON Path filter. It seems as if the additional preprocessing steps somehow affect JSON Path filter. (BTW, the same preprocessing steps have no issue when used with any other filtering action NOT including JSONPath).
Has anyone else got lost in the Twilight Zone like this? What really nags me is:
1. How come that the filter seems to work internally (as they produce a correct entry in Latest Data), but still the item is marked as failed and therefore never graphed? Where exactly the failure happens?
2. How come that once the item gets marked as failed, it cannot be restored to a valid version by falling back to aprevious JSONPath expression? Moreover, how can the cloned versions of an item still fail, and only recreating the item from scratch works again?
3. Is there some other "proper" way to use JSONPath filtering?
Just for the record, I eventually dropped JSONPath completely and used good-old (and ugly) Regex to parse the JSON data. I'm happy again.
I use Zabbix version 4.2.6 in RHEL 7. According to the documentation, this version should be able to preprocess item results using JSONPath. Nevertheless, in practice it is impossible to use it, except from implementing very basic filters.
Per instance, I created an item that receives the following JSON data:
[ {"result":"1", "charged":11111.00, "reported":22222.00, ...more fields here...}, ...more objects here... ]
I set the item to display 'Text' during the debugging phase, to avoid any conversion issues.
Now, assume that I want to extract the value of the field 'reported' when result==1. I did the following attempts on the way to find the magic filter for that:
Attempt 1:
JSON Path expr: $.[?(@.result=="1"]
Result: WORKS! It returns the object corresponding to result==1. How nice
Attempt 2:
JSON Path expr: $.[?(@.result=="1")].reported
Result: WORKS at first (returns [22222.00]) BUT it can fail later (see below)
Attempt 3:
JSON Path expr: $.[?(@.result=="1")].reported.first()
Result: FAILS! No error recorded in the logs.
Error message: Preprocessing failed for: [ {"result":"1", "charged":11111.00, "reported":22222.00, ...}, ... ]
I then removed the '.first()' part to return to the previous working version, but to my amazement, the item continued to fail with the same type of error. No matter how much I waited the item never recovered.
I cloned the item to create a new one - just changing the name & key - but I still got the same error! It seems as if the item, once got marked as failing, it will be considered failing for the rest of its life; similarly for its cloned selves.
Attempt 4:
I started again with a fresh new item. No problems this time! I'm back to the success I had on Attempt 2. Then I went on and added an extra action just for kicks (only to check the behavior, as the anticipated result is not yet meaningfully parsed). I added a simple trimming action:
JSON Path: $.[?(@.result=="1")].reported + Trim: []"
Result: FAILS! Same old story. Same even if I use Regex "\[(.*)\]" or any other random action. Zabbix does not allow me to workaround the issue by using extra steps!
Now, the very interesting bits, are that:
1. All JSON Path filters used are tested within the Zabbix item (in the preprocessing tab) and are SUCCESSFUL. Even third party testers prove the filters are OK.
2. Even in the failing cases, the 'Latest Data' page displays the CORRECT value! So indeed Zabbix went through all steps and created the expected result. Nevertheless, the row is marked with the above error and the item is never graphed.
3. When I add extra preprocessing steps - causing the preprocessing to fail - the error appears on the first step, i.e. the JSON Path filter. It seems as if the additional preprocessing steps somehow affect JSON Path filter. (BTW, the same preprocessing steps have no issue when used with any other filtering action NOT including JSONPath).
Has anyone else got lost in the Twilight Zone like this? What really nags me is:
1. How come that the filter seems to work internally (as they produce a correct entry in Latest Data), but still the item is marked as failed and therefore never graphed? Where exactly the failure happens?
2. How come that once the item gets marked as failed, it cannot be restored to a valid version by falling back to aprevious JSONPath expression? Moreover, how can the cloned versions of an item still fail, and only recreating the item from scratch works again?
3. Is there some other "proper" way to use JSONPath filtering?
Just for the record, I eventually dropped JSONPath completely and used good-old (and ugly) Regex to parse the JSON data. I'm happy again.