Help with aggregating prometheus metrics

[colin_leroy]

Hello,
I'm writing a template based on a Prometheus exporter (Micrometer).
I'm having difficulties with some items that generate lots of metrics. I'm only interested in totals, but I can't seem to figure a way to do it. For example, the http requests counters exported represent one metric per URI, status code, method, etc :

# HELP http_server_requests_seconds
# TYPE http_server_requests_seconds summary
http_server_requests_seconds_count{exception="None ",method="GET",outcome="SUCCESS",status="200",uri= "/actuator/prometheus",} 10.0
http_server_requests_seconds_count{exception="None ",method="GET",outcome="SUCCESS",status="200",uri= "/doit",} 19.0
http_server_requests_seconds_count{exception="None ",method="GET",outcome="SUCCESS",status="200",uri= "/**/favicon.ico",} 1.0
http_server_requests_seconds_count{exception="None ",method="GET",outcome="CLIENT_ERROR",status="404" ,uri="/**",} 1.0

I would like the total of http_server_requests_seconds_count.

If I create a Dependant item (depending on the master Prom item), and preprocess it using "Prometheus pattern" "http_server_requests_seconds_count", I get "cannot apply Prometheus pattern: data extraction error: multiple matching metrics found:"

Which is true. I've tried promql syntax like sum(http_server_requests_seconds_count) but it's not supported. I can only figure out two solutions, none of which seem clean to me:

• I can use a Discovery rule with item prototypes, get all those metrics into items, then add an item aggregating those with a grpsum[] function. I don't really want to do that because that would have Zabbix track hundreds of items and that wouldn't be very scalable.
• I can preprocess using Javascript and implement a very ugly hack, splitting the whole Prom item by line, space and '{' with such a function. I don't really want to do that either because well, it's really ugly :

var lines = value.split('\n');
var total = 0.0;
for (var i = 0; i < lines.length; i++) {
  var fields = lines[i].split(' ');
  var root_field = fields[0].split('{');
  if (root_field[0] == "http_server_requests_seconds_count")
    total += parseFloat(fields[1]);
}
return total;

Is there another, cleaner solution that I'm not aware of?

Thanks in advance,
Colin

[Semiadmin]

Preprocessing steps:
1. Prometheus to JSON: http_server_requests_seconds_count
2. JSONPath: $..value.sum()

[colin_leroy]

Thanks a lot, this makes much more sense I knew I was missing something!