Can Zabbix ingest Kafka topics

[peterharris]

I have been tasked with bringing Kafka topics into Zabbix, I have read documentation that is provided around integrations for Kafka and most of this seems to be around jmx metrics for the Brokers alerts being based around when a new master node is elected, etc,. However I am looking to bring the alerts from the topics into Zabbix front end so we can utilise triggers to be able to alert issues to the core network team.

Scenario
I have an FTTP EMS, running in docker over two nodes. This implementation also has Kafka, zookeeper and Logstash OOTB.

I can create topics on the nodes utilising postman API, and i am aware that I need to build Consumers for the topics.

How would i go about sending the data from the consumer into Zabbix?

Is this even possible?

We utilise Zabbix as our NMS across our technology stacks and would like to be able to monitor FTTP connections in the same way.

[Markku]

I don't really know about Kafka but my understanding is that it is somewhat similar to MQTT.

For an example implementation how to bring data published with MQTT in Zabbix, see mqtt-zabbix, my modified fork is here: https://github.com/markkuleinio/mqtt...mqtt-zabbix.py

Basic idea is that the code subscribes to the MQTT topic and then sends the data to Zabbix using Zabbix trapper items.

Markku

[peterharris]

Is this direct from an MQTT broker or from a consumer of some sort?

[Markku]

I maybe don't understand the question or the key concepts to answer properly, but looking at the code, it connects to the MQTT broker, subscribes to the configured MQTT topic, and processes any messages that someone else sends to the topic.

How would your expected process work in Kafka?

Markku

[peterharris]

Producer> Kafka Brokers> send topics to Consumers> mechanism to publish into Zabbix.

[Markku]

For me it sounds identical to the presented MQTT example case.

Markku

[BigSmooth]

Hi,
I am curious. What kind of data (or event) will your topic contain?
Olivier