JSONPath preprocessing matching regexp

[zabbixuserfk]

Hello,

Let's say we have a prometheus json like this (the real one is a lot bigger) :

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "kube_running_pod_ready",
          "namespace": "argo-rollouts",
          "pod": "argo-rollouts-6fdcf89f7c-mzxcl",
          "prometheus": "openshift-monitoring/k8s"
       },
       "value": [
         1641539963.261,
         "1"
       ]
     },
     {
       "metric": {
         "__name__": "kube_running_pod_ready",
         "namespace": "gitlab-runner",
         "pod": "gitlab-runner-controller-manager-56b77757c-82wxh",
         "prometheus": "openshift-monitoring/k8s"
       },
       "value": [
         1641539963.261,
         "1"
       ]
     },
     {
       "metric": {
         "__name__": "kube_running_pod_ready",
         "namespace": "openshift-cluster-csi-drivers",
         "pod": "ovirt-csi-driver-node-dlhz4",
         "prometheus": "openshift-monitoring/k8s"
       },
       "value": [
         1641539963.261,
         "0"
       ]
     },
     {
       "metric": {
         "__name__": "kube_running_pod_ready",
         "namespace": "openshift-toto-runner",
         "pod": "kub-runner-1-runner-7c5c45d454-dvv6g",
         "prometheus": "openshift-monitoring/k8s"
       },
       "value": [
         1641539963.261,
         "1"
       ]
     }
   ]
 }
}

1) I'd like to retreive all the pods with the "namespace" field matching the regexp "openshift-.*", like this :

[
  "ovirt-csi-driver-node-dlhz4",
  "kub-runner-1-runner-7c5c45d454-dvv6g"
]

But none of these seem to be working :

$.data.result.[?(@.namespace=~openshift-)].pod
$.data.result.[?(@.namespace=~"openshift-.*")].pod
$.data.result.[?(@.namespace=~"/^openshift-.*$/i")].pod

While, with an exact namespace it works fine :

$.data.result.[?(@.namespace=="openshift-cluster-csi-drivers")].pod

Result :

[
"ovirt-csi-driver-node-dlhz4"
]

How can I filter json fields matching a regexp ?


2) Another solution is that I can also retreive all the pods with a "value" :"0", but in that case, I need to display the namespace AND the pod on the same line so that I can filter again with a simple regexp.
But this :

$.data.result[?(@.value[1]=="0")].metric.[namespace,pod]

gives me this (namespace and pod on 2 lines) :

[
"openshift-cluster-csi-drivers",
"ovirt-csi-driver-node-dlhz4"
]

Any solution please ? Or is it a limitation of Zabbix jsonpath preprocessing ?

[ISiroshtan]

Hey mate.

1. Zabbix syntax can be slightly different then what you used to. Details of what is supported and what is not can be found here (for version 4.4 as that was the one I was testing your issue in... was to lazy to bring v.5 up from sleep in lab).

So to achieve what you ask following expression can be used:

$.data.result[?(@.metric.namespace=~"openshift")].metric.pod

Doing preprocessing test on your JSON gives me the following result:

["ovirt-csi-driver-node-dlhz4","kub-runner-1-runner-7c5c45d454-dvv6g"]

Which i believe is what you want.

2. Despite the fact that Zabbix uses regex without multiline mode on, you can still do multiline match.

I'm not exactly getting what you trying to match with regexp in this example, but lets say we have:

[
"openshift-cluster-csi-drivers",
"ovirt-csi-driver-node-dlhz4"
]

And we want to match 'ovirt-csi-driver-node-dlhz4' only if namespace contain 'openshift'. You can achieve it by:

openshift(.|\s)*"(.*)"

Where match group 2 will be the the pod name with no quotes.

Hope it helps!

[zabbixuserfk]

Hi ISiroshtan,

Thank you very much, this is indeed very helpful, but I should probably have begun with what I need as a final result.

This json is the result of a prometheus request which lists each pod status in $.data.result..value[1] ("1" : pod is running, or "0" : pod is not running). These pods ($.data.result..metric.pod) belong in different namespaces ($.data.result..metric.namespace), and I need to separate the pods in namespaces "openshift-*" from the pods in other namespaces.
So what I want is actually to list the non-running pods (= with $.data.result..value[1] == "0") from namespaces "openshift-*" on one hand, and the non-running pods from namespaces different from "openshift-*".
Is it possible to specify these 2 filters (value == "0" AND "namespace ~= openshift-*) in one jsonpath step or do I have to do it in several preprocessing steps ?

Let's consider my json is now this (with first pod in a non-running state too) :

{
"status": "success",
"data": {
"resultType": "vector",
"result": [
{
"metric": {
"__name__": "kube_running_pod_ready",
"namespace": "argo-rollouts",
"pod": "argo-rollouts-6fdcf89f7c-mzxcl",
"prometheus": "openshift-monitoring/k8s"
},
"value": [
1641539963.261,
"0"
]
},
{
"metric": {
"__name__": "kube_running_pod_ready",
"namespace": "gitlab-runner",
"pod": "gitlab-runner-controller-manager-56b77757c-82wxh",
"prometheus": "openshift-monitoring/k8s"
},
"value": [
1641539963.261,
"1"
]
},
{
"metric": {
"__name__": "kube_running_pod_ready",
"namespace": "openshift-cluster-csi-drivers",
"pod": "ovirt-csi-driver-node-dlhz4",
"prometheus": "openshift-monitoring/k8s"
},
"value": [
1641539963.261,
"0"
]
},
{
"metric": {
"__name__": "kube_running_pod_ready",
"namespace": "openshift-toto-runner",
"pod": "kub-runner-1-runner-7c5c45d454-dvv6g",
"prometheus": "openshift-monitoring/k8s"
},
"value": [
1641539963.261,
"1"
]
}
]
}
}

I was trying something like $.data.result[?(@.value[1]=="0")].metric.[namespace,pod] to list all the non-running pods with their namespace. A json parser like jsonpath.com gives me this,

[
"argo-rollouts",
"argo-rollouts-6fdcf89f7c-mzxcl",
"openshift-cluster-csi-drivers",
"ovirt-csi-driver-node-dlhz4"
]

but Zabbix doesn't seem to like it and I still would have to put the namespace and pod name on a same line so I can filter then by namespace with a regexp for example...

Preprocessing failed for: {"status":"success","data":{"resultType":"vecto r", "result":[{"metric":{"__name__":"kube_running_p...
1. Failed: cannot extract value from json by path "$.data.result[?(@.value[1]=="0")].metric.[namespace,pod]": unsupported construct in jsonpath starting with: "namespace,pod]"

[ISiroshtan]

Hey Mate,

Here is a quick fix to your expression:

$.data.result[?(@.value[1]=="0")].metric.["namespace","pod"]

I'm a bit pre-occupied now to read deeply into your end goal, so will get back to it somewhere later today or tomorrow. Sorry bout it.

[ISiroshtan]

Seems like what you want as end result is

$.data.result[?(@.value[1]=="0" && @.metric.namespace=~"openshift")].metric.pod

It should give you only pod names that are not running and have 'openshift' in it's namespace. With your JSON it gives me only:

["ovirt-csi-driver-node-dlhz4"]

[zabbixuserfk]

Hello,

It seems all good indeed, thank you very much !!
However, I wonder why $.data.result[?(@.metric.namespace=~"openshift")].metric.pod for example doesn't work on jsonpath.com ? Do you know any other json parser which would work more like Zabbix' so I could trust the tests I make please ?

[ISiroshtan]

I can only guess here, but i'd say that due to doing it through web-UI (PHP) and to make it compatible with normal Zabbix UI operations they had to introduce some changes in how they acceppt JsonPATH string. But truce to be told - the hell if I know

In regards to testing - I just use in-build pre-processing test feature. It might not give super detailed error message, but generally it's possible to realize where the error is.

Like with your error:

Preprocessing failed for: {"status":"success","data":{"resultType":"vecto r", "result":[{"metric":{"__name__":"kube_running_p...
1. Failed: cannot extract value from json by path "$.data.result[?(@.value[1]=="0")].metric.[namespace,pod]": unsupported construct in jsonpath starting with: "namespace,pod]"

it specifically points to portion where it can not read the syntax:

unsupported construct in jsonpath starting with: "namespace,pod]"

so I just had to open zabbix doc i linked initially and find the syntax supported by Zabbix and try it out in pre-processing test to verify.

[zabbixuserfk]

The problem here is that the json is so big that the testing tool cannot take it entirely (353 000 characters...) so I cannot rely on it...
But anyway, thank you very much for your help and have a good day !