Multiple metrics in Prometheus LLD

[xaban]

I created the following LLD:
- Name: Prometheus LLD
- Type: Dependent Item
- Key: lld_prom
- Master: Selected the HTTP Item which I created

Preprocessing:
- Prometheus to JSON: kube_deployment_status_replicas_available{namespac e="default",deployment=~".*"}

LLD Macros:
{#METRIC} -> $['name']
{#POD} -> $.labels['deployment']

Within this LLD I created an Item Prototype:
- Name: Metric: {#METRIC} - Pod: {#POD}
- Key: replicas_avail[{#POD}]

Preprocessing:
- Prometheus pattern: {#METRIC}{namespace="default",deployment="{#POD}"}

This works great and items are being created.

Now my question / problem:

As you can see, I am using the metric "kube_deployment_status_replicas_available", which returns the available replica's of a deployment.
The same Prometheus has another metric too, which is "kube_deployment_spec_replicas". This metric returns the amount of replica's that should be available.
I want have an Item Prototype too for this metric, so at the end I can create a Trigger Prototype, so I can do something: replicas_available>spec_replicas.

Is this possible, and how?

Thanks!

[Glencoe]

There is one theoretical problem with comparing discovered metrics this way. kube_deployment_status_replicas_available{} is not guaranteed to have the same combinations of namespaces/deployments as kube_deployment_spec_replicas{} at the protocol level. Maybe K8s takes care of this but this is something to keep in mind.

That being said, the idea is to create an LLD rule that creates JSON from one of the metrics but have two item prototypes (with fixed names instead of {#METRIC}) and one trigger for this rule.

Suppose, you need to compare incoming and outgoing traffic reported by node_exporter: node_network_receive_bytes_total vs node_network_transmit_bytes_total:

1. Create dependent discovery rule with Prometheus preprocessing:



In this case we need to map just one LLD macro:



2. Add two item prototypes:



... with standard "Prometheus pattern" preprocessing:



3. Add one trigger prototype that refers to both items:

[Glencoe]

4. Get the result:

[xaban]

Glencoe your help is much appreciated, that is exactly I was looking for!