JSONPath, CSV to JSON LLD Woes

[cwhite]

Hello Community -

I've read through previous posts and translated posts and still can not understand where I am going wrong. I've tried different bits and pieces to get a vendor API call (converted to JSON from csv) to LLD correctly and either it goes unsupported or I get random characters added to my item keys which then makes JSONPath not match. My current iteration has an API call through external check to a vendor url - that data is returned as CSV. I've tried converting within the script (PHP) to JSON and also using preprocessing CSVtoJSON within Zabbix (current). So currently after the API call my data looks as such

[
{
"PepVPN Name": "conn_to_PegaSDXPro-0-1",
"Download (MB)": "21848.00",
"Upload (MB)": "12417.00",
"Total (MB)": "34265.00"
},
{
"PepVPN Name": "conn_to_Peplink20",
"Download (MB)": "1404.00",
"Upload (MB)": "9082.00",
"Total (MB)": "10486.00"
},
{
"PepVPN Name": "conn_to_Peplink13",
"Download (MB)": "2383.00",
"Upload (MB)": "5312.00",
"Total (MB)": "7695.00"
},
{
"PepVPN Name": "conn_to_Peplink15",
"Download (MB)": "904.00",
"Upload (MB)": "1250.00",
"Total (MB)": "2154.00"
},
{
"PepVPN Name": "",
"Download (MB)": "281.00",
"Upload (MB)": "132.00",
"Total (MB)": "413.00"
},
{
"PepVPN Name": "conn_to_Peplink11",
"Download (MB)": "120.00",
"Upload (MB)": "199.00",
"Total (MB)": "319.00"
},
{
"PepVPN Name": "conn_to_Peplink10",
"Download (MB)": "153.00",
"Upload (MB)": "156.00",
"Total (MB)": "309.00"
},
{
"PepVPN Name": "conn_to_Peplink12",
"Download (MB)": "103.00",
"Upload (MB)": "142.00",
"Total (MB)": "245.00"
},
{
"PepVPN Name": "conn_to_Peplink18",
"Download (MB)": "78.00",
"Upload (MB)": "79.00",
"Total (MB)": "157.00"
},
{
"PepVPN Name": "conn_to_PegaFusionHub2",
"Download (MB)": "48.00",
"Upload (MB)": "47.00",
"Total (MB)": "95.00"
}
]

Now end goal is to have this JSON (which changes based on end points connected to the VPN), to walk the return daily, create a unique key based on "PepVPN_Name" and for every "PepVPN Name" an Item Prototype for "Download, UL, Total". I made a discovery rule (dependent item from php external check) with no preprocessing and one with JSONpath $.[*] - either way runs with no error logs. Both have LLD Macro for {#SPOKENAME} = "PepVPN Name" to use in item prototypes. Testing a "Total" prototype I have unique key[{#SPOKENAME}], preprocessing listed below and using the JSON above i get the correct return value with 'TEST". Yet on the host I get nothing created nor any errors to see what is wrong. I've tried so many permutations that I'm frankly lost as to what to try next.



Any direction is greatly appreciated!

The half working one is same discovery but after it creates the items but they go unsupported with a JSONpath error. From what I can tell the " is my problem which shows up in the unique key[{#SPOKENAME}], yet the display name in the item shows correctly/differently . Nothing ever shows up in host latest data even though items are created/enabled/unsupported.

[cwhite]

As a follow-up - THIS IS DRIVING ME CRAZY -

So I thought possibly the issue was how the LLD MACRO is being extracted, and I assume this is specific with how external scripts parse JSON (as dependent items) versus how zabbix agent works. I went down the path of the JSON looked fine compared to similar items I have using zabbix_agent to parse JSON. Agent seems to handle quotes, space etc just fine and pull just the string values. My external php seems to work fine running manually both on cli and 'test' within Zabbix - yet the item is created key["["conn_to_Peplink7"]"] (as an example) - where [" "] are randomly inserted into the key and that is the value the MACRO is passing to JSONPath and thus failing. Looking at the item description for same key value those escape characters are also present Spoke ["conn_to_Peplink7"] Total Data.

Now I thought the JSON might be an issue since the path names contained spaces, I used php within my script to replace the space with '_' and still get same return values with updated JSON as below

{

"1": {

"PepVPN_Name": "conn_to_Peplink10",

"Download_(MB)": "733.00",

"Upload_(MB)": "4369.00",

"Total_(MB)": "5102.00"

},

"2": {

"PepVPN_Name": "conn_to_Peplink20",

"Download_(MB)": "435.00",

"Upload_(MB)": "3295.00",

"Total_(MB)": "3730.00"

},

"3": {

"PepVPN_Name": "conn_to_PegaSDXPro-0-1",

"Download_(MB)": "233.00",

"Upload_(MB)": "507.00",

"Total_(MB)": "740.00"

},

"4": {

"PepVPN_Name": "conn_to_Peplink11",

"Download_(MB)": "76.00",

"Upload_(MB)": "77.00",

"Total_(MB)": "153.00"

},

"5": {

"PepVPN_Name": "conn_to_Peplink12",

"Download_(MB)": "68.00",

"Upload_(MB)": "67.00",

"Total_(MB)": "135.00"

},

"6": {

"PepVPN_Name": "conn_to_Peplink310-1",

"Download_(MB)": "55.00",

"Upload_(MB)": "54.00",

"Total_(MB)": "109.00"

},

"7": {

"PepVPN_Name": "conn_to_Peplink18",

"Download_(MB)": "49.00",

"Upload_(MB)": "56.00",

"Total_(MB)": "105.00"

},

"8": {

"PepVPN_Name": "conn_to_PegaFusionHub2",

"Download_(MB)": "31.00",

"Upload_(MB)": "31.00",

"Total_(MB)": "62.00"

},

"9": {

"PepVPN_Name": null

}

}

So does anyone know how I can get these values to extract correctly to LLD MACRO without these extra escape characters????

I need to parse the JSON and take PepVPN_Name value as unique LLD key and then create an item for each LLD MACRO {PepVPN_Name} for its Download, Upload, Total values.

[cwhite]

cyber you are a life saver - I knew it had to do with escape char and spaces, just didn't take it far enough. I did have to add preprocess to the Discovery though in addition, JSON path $.* or I would get discovery error on UI talking about old Zabbix version and 'data' key. I had to do some conversion to Float as well so I can graph the data flow as Zabbix treated it as strings.

Again thank you sooooo much!

I'm actually going to apply this to another call I have as items and convert it to discovery as well (for the end points connected to the VPN Hubs).

[cyber]

json from your first post.. format it without spaces in keys...
Something like...
{ "PepVPN": "conn_to_Peplink10", "Download": "153.00", "Upload": "156.00", "Total": "309.00" }, etc.
Create your discovery (pepdisco)... dependent item, master item is the one, what gives you complete json (pepmain in my example) (I just used already complete json from your post)... In LLD macros add "{#PEPNAME}" => "$.PepVPN"
Create item prototypes.. dependent items, master item is same as for discovery. Name: "{#PEPNAME}" download", key: download[{#PEPNAME}], type: Numeric(float) ... add preprocessing jsonPath => $.[?(@.PepVPN=='{#PEPNAME}')].Download.first()
Repeat for upload and Total.. replacing jsonpath accordingly...
should end up with nice colorful list like this...

and latest data should show you

Each time your main item runs, all disco is run also and existing items get their values... if something is not there any more, those are marked as "not discovered any more"

[cyber]

You are welcome.....