Hi, I am trying to set up Kafka lag monitoring in Zabbix and I am struggling with LLD.
My steps taken:
1. I created the userparameter which runs bash script which then runs kafka-consumer-groups.sh
2. That produces following JSON:
3. In discovery rule, I set up preprocessing:
4. Two LLD macros:
When I click Test here I just see whole JSON, not subsequent values in macros
Also, when I click Test on main discovery rule page I also see whole JSON without subsequent extracted values.
I am trying to set up item prototype from this that would be filled with lag values, but not sure if I am doing something wrong in previous steps or how to really proceed - tried to reuse the same userparameter in item prototype I used in discovery rule but nothing really gets created.
What is important - my userparameter is a script that does not take any parameters, it passively gets data and that's what I am trying to operate on. I prefer not do call Kafka for each group each 1-2 minutes - cluster I operate on is quite big and polling data from it takes too much time and resources if it was spammed constantly.
My steps taken:
1. I created the userparameter which runs bash script which then runs kafka-consumer-groups.sh
UserParameter=kafka.consumer.group.lag,/usr/lib/zabbix/externalscripts/kafka-lag.sh
Code:
#!/bin/bash
/opt/kafka/bin/kafka-consumer-groups.sh --bootstrap-server 10.1.1.1:9092 --describe --all-groups 2>/dev/null | awk -v OFS=',' 'NR > 1 && $1 != "" && $1 != "GROUP" && $6 ~ /^[0-9]+$/ {lag[$1]+=$6} END {print "["; first=1; for (group in lag) {if (!first) print ","; first=0; print "{\"group\":\"" group "\", \"global_lag\":" lag[group] "}"} print "]"}'
Code:
[
{"group":"multi-threaded-consumer-fast-2-group", "global_lag":0}
,
{"group":"single-threaded-consumer-1-group", "global_lag":0}
]
4. Two LLD macros:
When I click Test here I just see whole JSON, not subsequent values in macros
Also, when I click Test on main discovery rule page I also see whole JSON without subsequent extracted values.
I am trying to set up item prototype from this that would be filled with lag values, but not sure if I am doing something wrong in previous steps or how to really proceed - tried to reuse the same userparameter in item prototype I used in discovery rule but nothing really gets created.
What is important - my userparameter is a script that does not take any parameters, it passively gets data and that's what I am trying to operate on. I prefer not do call Kafka for each group each 1-2 minutes - cluster I operate on is quite big and polling data from it takes too much time and resources if it was spammed constantly.
Comment