Discussion thread for official Zabbix Template RabbitMQ

[Semiadmin]

There is a weird thing with this template. Dependent LLD rules execute every time master items get JSON from Rabbit API, i.e. every minute. It's not good for performance. Maybe add 2-step preprocessing to the LLD rule:
1. JS: return JSON.stringify(JSON.parse(value),['type','name','vhost'])
(for Exchanges discovery, ['node','name','vhost'] for Queues discovery)
2. Discard unchanged with heartbeat 1h

[czechsys]

Can you please add to default template this macro and update all references to it?:

#allow set to http or https scheme

Macro: {$RABBITMQ.API.SCHEME}
Value: http (default)
Desciption: The http scheme of RabbitMQ API endpoint
Example item:

RabbitMQ: Get node overview:

web.page.get["{$RABBITMQ.API.SCHEME}://{$RABBITMQ.API.USER}:{$RABBITMQ.API.PASSWORD}@{$RA BBITMQ.API.HOST}:{$RABBITMQ.API.PORT}/api/overview"]

[c.h.]

I'm currently using Zabbix 5.0.16 (the long-term support version), and am also having trouble with "Template App RabbitMQ node by Zabbix agent - active" template.
The template for my version is here: https://git.zabbix.com/projects/ZBX/...Ftags%2F5.0.16

At any rate, the "RabbitMQ: Get node overview" and "RabbitMQ: Get queues" items work, but the "Queues discovery" discovery rule doesn't. I'm not sure how to debug this, as I'm a Zabbix beginner.

[c.h.]

Apparently the secret to getting it working is to post a message to this forum, because right after doing that, I got it working!

I had removed the preprocessing step from something, and when I put it back, /var/log/zabbix/zabbix-server.log showed that my rabbitmq.queue.* keys were suddenly available!

Edit: Oh... while writing this, they "became not supported" again. :-(

[c.h.]

Is there an easy way to reinstall the rabbitmq module, erasing my errors?

[[email protected]]

c.h. - your problem may be related to the problem discussed here: https://groups.google.com/g/rabbitmq.../c/N0EgrLn55XQ - apperantly the template uses HOST.NAME macro in the URL:

'{$RABBITMQ.API.SCHEME}://{HOST.CONN}:{$RABBITMQ.API.PORT}/api/nodes/{$RABBITMQ.CLUSTER.NAME}@{HOST.NAME}?memory=true'

- while your rabbitmq can use short hostnames and that causes problem in rabbit. I'm investigating this now.

[c.h.]

Actually, my first problem ended up being that I was using the wrong username; amazing how that can foul things up!

The current problem is that I'm trying to alarm on queues that have too many messages in them, and I'm getting this error:

Preprocessing failed for: [{"arguments":{"x-dead-letter-exchange":"dead-letter","x-max-length":1000000,"x-max-length-bytes"...
1. Failed: cannot extract value from json by path "$[?(@.name == "["control"]" && @.vhost == "["/"]")].messages_ready.first()": no data matches the specified path

[c.h.]

How does one debug this sort of thing?

I may have made changes to it while trying to solve the first problem, which probably created the second problem, which led to my other question (shown above) on "how to reinstall the rabbitmq module".

I'm used to editing a file, pushing it with git, and seeing how it runs in Jenkins. It's so nice to be able to roll back to a previous version, and to be able to see all of the code involved.

Any advice would be welcome on this point!

Thanks

Edit: The "RabbitMQ: Get nodes" raw item is now working, simply because of the issue described above (the "{HOST.NAME}" macro is the full host name, but rabbitmq needs the short host name). I followed their advice to create `{$RABBITMQ.CLUSTER.HOSTNAME}` macro, which worked, and then tried every macro function I could think of to make it not need a custom macro for every host:

web.page.get["http://web:[email protected]:15672/api/nodes/rabbit@{HOST.HOST1}?memory=true"]
web.page.get["http://web:[email protected]:15672/api/nodes/rabbit@{HOST.NAME1}?memory=true"]
web.page.get["http://web:[email protected]:15672/api/nodes/rabbit@{{agent.hostname}.regsub(\"[.].*\",\"\")}?memory=true"]
web.page.get["http://web:[email protected]:15672/api/nodes/rabbit@{{HOST.HOST1}.regsub(\"[.].*\",\"\")}?memory=true"]
web.page.get["http://web:[email protected]:15672/api/nodes/rabbit@{{HOST}.regsub(\"[.].*\",\"\")}?memory=true"]

(What you get if the macro isn't defined:)
web.page.get["http://web:[email protected]:15672/api/nodes/rabbit@{$RABBITMQ.CLUSTER.HOSTNAME}?memory=true"]

Edit2: Apparently if you set

RABBITMQ_USELONGNAME=true

in RabbitMQ's environment variables, then RabbitMQ will use the full host name for node names, working around this macro limitation.

Edit3: But that environment variable (or the cli option `-l`) has to be specified when the cluster is being configured, or else it has no effect. :-(

[c.h.]

Ok, I've got it working (mostly)!

• One blocker was that a number of items in the RabbitMQ application were set to "Zabbix Agent", not Zabbix Agent (active): only active agents can work in our network.
• The LLD macros on the Queues discovery rule; they needed to have .first() appended (e.g. {#NODE} = "$..node.first()"), otherwise they would contain lists instead of strings.
• Cosmetically, the item prototypes now need some brackets to be added to their names to make them more readable
  • e.g. 'RabbitMQ: Queue "//dead-letter"' -> 'RabbitMQ: Queue [/][dead-letter]'
• Some items had the same name but different values
  • e.g. 'RabbitMQ: Queue ".....": Messages delivered" was for
    • "Count of messages delivered in acknowledgement mode to consumers" and
    • "Sum of messages delivered in acknowledgement mode to consumers, in no-acknowledgement mode to consumers, in acknowledgement mode in response to basic.get, and in no-acknowledgement mode in response to basic.get"
• It was also helpful to rename the items so that they have unique suffixes: when graphing the number of messages in every queue (for instance), putting 'Messages' in the 'Name' search field of the "Latest data" search form matched every item, so changing the name to 'Messages in queue' helped cut that down tremendously.
• Then because most of the item prototypes also had 'per second' keys, 'in total' was added to the ones that weren't 'per second', again to help with searching.

The only thing left to do is to configure a trigger for queues that are too large, where 'too large' depends on the name of the queue (e.g. the dead-letter queue should be empty all the time, others can be larger).

I learned a lot about how to debug the Zabbix server configuration, and look forward to version 6.0.0 when it comes out next month. I'm sure it'll be even more user-friendly!

[vkhaliev]

• Queues discovery rule is dependent, so you can execute it manually by execution of master item.
• If discovery rule doesn't work check macros values. Macro {HOST.NAME} used as a node name at filters in queues discovery. It must be the same as node name in RabbitMQ. Actually this macro get data from visible name of the host, but if it is not specified host name will be used instead. So the simplest way is to configure visible name or host name in zabbix.
  Also you can setup your own macro like it was recommended in previous comments, but in this case you will need to change filter in queues discovery.
  Macro {HOST.NAME} also used at the URL of "Get Nodes" item, so if you use your own macro you need to change this key too. Otherwise "Failed to fetch nodes data" trigger status can be "problem".
• Macro {$RABBITMQ.API.SCHEME} will be added to RabbitMQ by agent template in https://support.zabbix.com/browse/ZBX-20562.
• Item names will be fixed in https://support.zabbix.com/browse/ZBX-20562.
• Trigger expression "Too many messages in queue" already has context macro with queue name, so you can configure threshold for every queue. (For example: {$RABBITMQ.MESSAGES.MAX.WARN:"dead-letter"}, value "0").
  Link to documentation about context macros: https://www.zabbix.com/documentation...macros_context

[c.h.]

Wow, thanks for this information, I'll give it a try later today!

[keitht]

I just upgraded from Zabbix 5.4 to Zabbix 6.0 and I'm trying to update my rabbitmq related templates, which are probably from the original 5.0 install. I have had multiple RMQ clusters being monitored via the "Template App RabbitMQ cluster by Zabbix Agent" and "Template App RabbitMQ node by Zabbix Agent." Looking at the templates folder in the Zabbix repo, I only see one of these two templates, RabbitMQ cluster by Zabbix agent, at app/rabbitmq_agent/. When I tried to update the cluster template, it failed saying 'Graph with name "RabbitMQ: Node status" already exists" and I suspect this may be related to my existing node template. The node template is still listed on the Zabbix website on the rabbitmq integrations page, right under the cluster template. Why doesn't the node template show up in the templates folder of the zabbix repo?

Thanks
Keith

[j.h.]

I seem to be running into an issue with the Node template's discovery, I see that I get the information through the master item of Get Queues, but after that there's no information for the discovery items/triggers I have created. I have made sure all the macros are correct for the host and for the LLD macros. Is there something else I should be doing? The discovery items for the Cluster template are working like a charm. Any help would be greatly appreciated.

[vkhaliev]

Template file from zabbix repo contains both templates (cluster and node).
If you have problem with import, you can try to rename existing templates.

[vkhaliev]

j.h., can you specify your problem? You have no one discovered item or discovered items has no data?
If you are using default template and discovery rule doesn't create items it can be related with macros and host name in zabbix, because it is used in filters of discovery rule (node name must to be matched with "{$RABBITMQ.CLUSTER.NAME}@{HOST.NAME}").
For example: if i have cluster name "rabbit" and node "rabbit-node3" with some queues, then the name of the node in rabbit interface is rabbit@rabbit-node3. Therefore host macro {$RABBITMQ.CLUSTER.NAME} must be set to rabbit and visible name of the host in zabbix or host name (if visible name is empty) must be rabbit-node3.
If discovered items has no data and became unsupported it is needed to check the error message for this unsupported items.