Discarding JSON with specified attribute value

[vincemartinsmith]

Hi folks,

I have an interesting issue (well I think it's interesting
I have a LoRaWAN device which sends data packets in two different formats. In Zabbix I want to differentiate between them and discard one format.

As messages come in via MQTT the value gets set in the master item.
Dependent items break out the various metrics of interest.
BUT some master item values (where fPort == 0 have nothing in Object ie {}

In simplified form... I want to discard messages where fPort is 0
Otherwise other dependent items get grumpy because Object doesn't have the expected fields.

{
"fPort": 0,
"Object": {},
}

or the one I want to keep:

{
"fPort":85,
"Object":{ "value1":1234, "value2":3456}
}

Dependent objects look for value1 value2 and report "no data matches the specified path" when fPort 0 messages arrive.


At this point it looks like I have to add a custom error handler to catch this on each and every dependent field and subsequent calculated fields.
Can anyone point me in the right direction ?

TIA.
Vince.

[Brambo]

Preprocessing failed for: {"applicationID":"5","applicationName":"MyDevic e", "deviceName":"UC3006378","deviceProfileName" :...
1. Failed: cannot extract value from json by path "$.object": no data matches the specified path

Did you check the "custom on fail" checkbox?
If you do you get the option on how to proceed, Either set a value or maybe discard it all, or set item to a certain error.
When it's uncertain if data is always there it's good practice to use this options in your template items.

[Brambo]

Looking at your data example the easiest way is create the dependent items with a discovery rule and use filter in discovery to just ignore the fPort 0
But as we don't see the full json-construct this might be slightly more difficult than the example (like are they send over 1 by 1 or all in 1 json etc etc)

[vincemartinsmith]

Thanks for responding Brambo
I am still learning these aspects of Zabbix so please bear with me.

Perhaps I misunderstood the function of discovery and associated filters but I'd have thought the "discovery" would be at the topic level rather than payload content.
Sure we can filter out one message based on the fPort value and thus "not discover" an item based on that message - but the next message will likely still have good content so would "discover" the item anyhow.


More background:

This data originates from a Chirpstack LoRaWAN server - I am also investigating further what I might be able to do at the codec level - thus far I've not managed to actually discard the bad packets so attempting to do it inside Zabbix.

I'm thinking at this point perhaps some javascript might give me the opportunity to discard a packet that is considered 'bad' ? ... but not found any method of doing so that way either - everything I've tried results in a NULL or EMPTY master item in which case all dependent items report errors.

To explain more fully...

The fPort I mention actually comes inside each JSON object received by the Master Item - so both fPort = 0 and fPort = 85 will still be processed by whatever logic is used at the item level.
Message examples below of the full JSON payload, FWIIW - as you can see the actual BAD messages are missing the "object" part of the message. Data is either NULL or empty object {} depending on how I code the codec.


The master item is defined with a key like the following:

mqtt.get["tcp://<my mqtt broker>:1883","application/5/device/24e124445c400329/event/up","mqttuser","password"]

PreProcessing:
Step 1: JSONPath
Parameters: $.object

If fPort == 0 then data contains nothing, so $.object doesn't exist thus throws a wobbly.

The error on the master item is:

Preprocessing failed for: {"applicationID":"5","applicationName":"MyDevic e", "deviceName":"UC3006378","deviceProfileName" :...
1. Failed: cannot extract value from json by path "$.object": no data matches the specified path

which completely makes sense.

I've tried adding additional steps to 'detect' a bad message, but always a NULL seems to get through to the dependent items.
Ideally I just want to discard and not process the message so the dependent items just see the last valid value.



[NORMAL MESSAGE - This is what shows up at the Master Item - on a good day]
{
"applicationID":"5",
"applicationName":"MyDevice",
"deviceName":"UC3002915",
"deviceProfileName" :"UC 300",
"deviceProfileID":"0cefa4dd-7147-414b-9c28-e934d1f6d19c",
"devEUI":"24e124445c192915",
"rxInfo" :[{"gatewayID":"24e124fffef1bcbd","uplinkID":"234 892 fb-6128-4f88-8e2d-237041c9048b","name":"Hastings-72","time":"2025-03-07T04:06:04.655069Z","rssi":-119,"loRaSNR":-18,"location":{"latitude":-39.0,"longitude":174.0,"altitude":401}}],

"txInfo":{"frequency":922200000,"dr":0},
"adr": true,
"fCnt":2351,
"fPort":85,
"data":"AwAABwEBCAEBDAJ TB QAADQIuAAAADgIQAAAACmdtAQ==",
"object":{"adc_2":13. 63,"adv_1":0.46,"adv_2":0.16,"gpio_in_1":0,"gpio_ o ut_1":1,"gpio_out_2":1,"pt100_2":36.5}
}

[BROKEN MESSAGE VERSION 1]
{
"applicationID":"5",
"applicationName":"MyDevice",
"deviceName":"UC3002915",
"deviceProfileName":"UC 300",
"deviceProfileID":"0cefa4dd-7147-414b-9c28-e934d1f6d19c",
"devEUI":"24e124445c192915",
"rxInfo" :[{"gatewayID":"24e124fffef35f55","uplinkID":"1a5 72d 78-b76b-444b-804a-189080cfa826","name":"Palmer-77","time":"2025-03-07T04:14:44.254771Z","rssi":-119,"loRaSNR":-21.5,"location":{"latitude":-39.0,"longitude":174.0,"altitude":548}}],
"txInfo":{"frequency":922800000,"dr":0},
"adr":true,
"fCnt":4514,
"fPort":0,
"data":null
}

[BROKEN MESSAGE VERSION 2]
{
"applicationID":"5",
"applicationName":"MyDevice",
"deviceName":"UC3002915",
"deviceProfileName":"UC 300",
"deviceProfileID":"0cefa4dd-7147-414b-9c28-e934d1f6d19c",
"devEUI":"24e124445c192915",
"rxInfo" :[{"gatewayID":"24e124fffef35f55","uplinkID":"1a5 72d 78-b76b-444b-804a-189080cfa826","name":"Palmer-77","time":"2025-03-07T04:14:44.254771Z","rssi":-119,"loRaSNR":-21.5,"location":{"latitude":-39.0,"longitude":174.0,"altitude":548}}],
"txInfo":{"frequency":922800000,"dr":0},
"adr":true,
"fCnt":4514,
"fPort":0,
"data":{}
}

[Brambo]

Preprocessing failed for: {"applicationID":"5","applicationName":"MyDevic e", "deviceName":"UC3006378","deviceProfileName" :...
1. Failed: cannot extract value from json by path "$.object": no data matches the specified path

Did you check the "custom on fail" checkbox?
If you do you get the option on how to proceed, Either set a value or maybe discard it all, or set item to a certain error.
When it's uncertain if data is always there it's good practice to use this options in your template items.

[vincemartinsmith]

Brambo, apologies for taking so long - completely dropped off my radar.
Thanks for your pointer there ... it was that simple

Marked as solution.

Regards
Vince.