Zabbix LLD performance - no data

[Amiram]

Hey all,

I'm testing the ability of the to "scrape" Prometheus http data.

I have created a template containing and Item and LLD rule as follow:

• Item - scrape the relevant Prometheus page (No history)
• LLD - depended to the Item (keep lost resources period 3days)
• LLD - Item - depended to the item

With a bit of an item preprocessing (at the lld item) I have successfully converted each of the Prometheus metrics to a Zabbix item.

This is nice for a single serve, the results as expected.

But when adding this template to all of my servers something is do terribly wrong --> all the items have been created BUT no data extracted NONE. --> no graphs!!
The first thing I have noticed that my LLD worker is way too high 100% --> added several more until it stabilized on 50%.
And still the problem remains.

My Environment setup is:
One Server and Two Proxy.
The server does not monitor anything -> everything is via the proxies.
The NVPS is 1380 with an ~1,000,000 Items (including all of the Prometheus data) for 1300 hosts

• Proxy 1 monitor 1100 servers (no Prometheus scraping here)
• Proxy 2 monitor 200 servers and scrape the Prometheus data of those 200 servers

The Zabbix versions 5.2 (Server and Proxy)
The Zabbix server running on 32 Cores with 386GbRAM on MySQL database.

Please HELP.

It seems that all of the preprocessing work is running at the Proxy which cause to high CPU load

P.S
Proxy 2 should monitor 2K servers at the end.