zabbix_sender: resolution of item key names

[Yello]

Hi,
At the moment (1.8.5) zabbix sender can't correctly resolve item key names which make use of macros. Is it in the current plan to enable this type of behaviour?


Regards,
David

[richlv]

could you please post an example on what you do and what you expect to happen ?

[Yello]

Hi,
Ok, here's an example. We have services/hosts spread across dev, qa and live with each configured in zabbix. Some items in a common template get data posted from zabbix_sender as part of batch monitoring...

The item key format I wanted to use to handle this was:

<service category>.<type>[Domain,Environment,JobName]

where:

Domain = Dev, QA or Live
Environment = A or B

This is consistent with the way I form trapper keys generally.

Each host runs both an A and B software environment so that's not an issue. However, I when I tried setting the domain portion of the key to a host macro I realised that zabbix_sender didn't see the value of the macro but rather it's name ({$DOM}), which couldn't work.

This will mean that I either have to create more item keys, many of which will be redundant depending on the host (dev, qa or live), create templates per domain(to avoid redundancy at the host level) or remove the "Domain" requirement from the item key.

As it turns out I've removed the domain argument from the item key and I'm ok with that. But having zabbix_sender able to resolve host macro values used in trapper item keys strikes me as sensible functionality to implement for the future.


Regards,
David

[richlv]

could you show an example full item key ? i believe you should be able to simply send values to that key as-is.

resolving macros by sender isn't technically feasible, but, even if it was, i don't think it would accomplish anything. you have to specify a host when sending values using sender. a host may have only one item with the same key. so that key is unique per host, and macro value does not actually concern the sender.

i hope that's at least somewhat understandable, and that i at least somewhat understood you

[Yello]

Hi,
Sorry for the delayed reply...

Here's an example of the type of key I'm referring to:

ingestion.refdata[Dev,A,ContentVerticals]

This key, like all my items, is template based. So a host, whether it's development, QA or live will be associated with the same template. With agent items I handle context related issues with macros. So the above key would become:

ingestion.refdata[{$MACRO},A,ContentVerticals]

...the macro vale would be set at the host level to provide host specific context to the item.

I understand what you say about zabbix_sender not being able to resolve a macro name to a value. So why not have zabbix expose an item key containing macros, such as this, to zabbix_sender as:

ingestion.refdata[<MACRO_VALUE>,A,ContentVerticals]

and not:

ingestion.refdata["{$MACRO}",A,ContentVerticals]

which zabbix_sender can never comply with.

As I say, I've got round this by removing the operational domain reference from the item key. But seeing as macros can be inserted into item keys it makes sense that they be made compatible with trapper items.


Regards,
David

[richlv]

ingestion.refdata["{$MACRO}",A,ContentVerticals]

which zabbix_sender can never comply with.

could you please explain why sender can not comply with that ?

[Yello]

Hi,
From your answer I infer that you believe that macro-enabled of item keys shouldn't be a problem for zabbix_sender. Consequently, I've tested this again and found something out that wasn't clear to me previously...

At my site we use a lot of nested templates. If an item macro is set in one of the nested templates it appears that that the intended item key name isn't exposed to zabbix_sender (the behaviour I described). However, if the macro is set at the host level then it is.

So, it looks like macros do work properly with zabbix_sender so long as the macro is set in the correct place. Thankyou for taking the time to discuss this with me


Regards,
David

[richlv]

nested templates and usermacros was a problem till... some of the 1.8 versions. latest version should work just fine. so if you are not using the latest version, you might want to wait for 1.8.9 and try again with that nested setup

[Spectroman]

Macro usage bug with zabbix_sender (?)

So, analyzing the problem of zabbix_sender to a key item with a {$MACRO} I found the following:

zabbix_sender -V
Zabbix Sender v1.8.7 (revision 21394) (1 September 2011)
Compilation time: Sep 2 2011 16:21:42

Zabbix server is from same version.

{$DB_NAME} => MPP001

(Macro configured at the host profile level)

KEY 1: end_date[<$DB_NAME>] #according with suggestions we've read around
KEY 2: errors[{$DB_NAME}]
KEY 3: start_date

Test with Key 1, with key value matching contents of {$DB_NAME}:

zabbix_sender -vv -c /etc/zabbix/zabbix_agentd.conf -s yblablablabla -k 'end_date[MPP001]' -o "blabla"
zabbix_sender [32579]: DEBUG: answer [{
"response":"success",
"info":"Processed 0 Failed 1 Total 1 Seconds spent 11.647334"}]
Info from server: "Processed 0 Failed 1 Total 1 Seconds spent 11.647334"
sent: 1; skipped: 0; total: 1

--> Attention to 11secs spent to send the information, for me, means that it found the item but was unable to finish the process for some unknown reason.

Test with Key 1, with key name incomplete (no Macro value / name):

zabbix_sender -vv -c /etc/zabbix/zabbix_agentd.conf -s yblablablabla -k "end_date" -o "blabla"
zabbix_sender [32348]: DEBUG: answer [{
"response":"success",
"info":"Processed 0 Failed 1 Total 1 Seconds spent 0.000049"}]
Info from server: "Processed 0 Failed 1 Total 1 Seconds spent 0.000049"
sent: 1; skipped: 0; total: 1

--> It rejects the sender very quickly, showing he identifies the wrong key name.

NOW the funny part:

Key 1, sending the information with the key name as-is:

zabbix_sender -vv -c /etc/zabbix/zabbix_agentd.conf -s yblablablabla -k 'end_date[<$DB_NAME>]' -o "blabla"
zabbix_sender [32615]: DEBUG: answer [{
"response":"success",
"info":"Processed 1 Failed 0 Total 1 Seconds spent 0.000061"}]
Info from server: "Processed 1 Failed 0 Total 1 Seconds spent 0.000061"
sent: 1; skipped: 0; total: 1

--> IT WORKS! but uselessly, cos the Macro became useless.

Key 2, same funny business:

zabbix_sender -vv -c /etc/zabbix/zabbix_agentd.conf -s yblablablabla -k 'errors[{$DB_NAME}]' -o "99"
zabbix_sender [32684]: DEBUG: answer [{
"response":"success",
"info":"Processed 1 Failed 0 Total 1 Seconds spent 1.334013"}]
Info from server: "Processed 1 Failed 0 Total 1 Seconds spent 1.334013"
sent: 1; skipped: 0; total: 1

Key 3, obviously works normally.

---

Shall we declare this a bug, or there were improvements in recent versions?

[Spectroman]

[update]

Updating zabbix_server to 1.8.10, problem still the same.

[richlv]

an item key is a unique identifier. there is one key only per data source, per item.
you must create items before you can get data in them.

could you, please, explain what exactly makes macros useless ?

[Spectroman]

Macro substitution

Maybe my explanation was not sufficient. I will try to be as forthcoming as possible.

- When you use a (user macro) {$MACRO}, its because you expect that you can use them in items and triggers, according to Documented feature in: http://www.zabbix.com/documentation/...s#user_macros;

- From there, you can expect that every time that you refer to a item which contains a "macro substitution" in a template, its because you can send a dynamic value to be substituted in place of the user macro, behavior that properly occurs in all the other parts that this function is expected to happen, or, excepting when using zabbix_sender;

- In the case of Zabbix_sender, when zabbix server is receiving such key that contains a user macro, the server cannot translate to a value and vice-versa, therefore rendering the key with a user macro useless, instead, zabbix server is only recognizing the key with the user macro as if it was a literal and not as a variable; -- according to the documentation at the link cited above: "In other words, if a macro does not exist for a host, Zabbix will try to find it in host templates of increasing depth. If still not found, a global macro will be used, if exists. " -- Therefore, the macro in my examples are in a host template, and they should have been found somehow and substituted by the value I am sending, as they are meant the create a dynamic substitution and able to cover different user macros across different hosts with a specific template.

- In the logic described above I can understand the following -- which actually is true in all other cases excepting for zabbix_sender --

a) host profile: contains a user macro called {$DB_NAME}=>MPP001
b) item key1: contains item zstatoracle[-d {$DB_NAME} -q query.props -o tblspaceWarn] and it is from type "External check"
c) the command zstatoracle, from template attached to such host (a) , receive in the parameter -d the value from user macro {$DB_NAME} properly substituted to its value (MPP001) during the execution of such external command.

But, the other way around does not happen, as if:

a) host profile: contains a user macro called {$DB_NAME}=>MPP001
b) item key1: contains item end_date[{$DB_NAME}] and it is from type "Zabbix Trapper"
c) Calling the zabbix_sender with the key end_date[MPP001] -- which matches the value from user macro {$DB_NAME} in host's profile, gives an error because zabbix is unable to find the proper substitution to such user macro and match a valid key with the user macro, instead it can only see a literal key value... thus, rendering the user macro in this case, useless.

- Concluding that for now, either the documentation should state that user macros are not meant to be used via zabbix_sender (or zabbix trapper) type of items (thing I could not find anywhere) or there is a bug when combining this configuration: Zabbix_sender + key with user macro

Is that clear enough?

[richlv]

right. but functionally there can be only one such key per host - for example, if you have a key :

key[$MACRO}]

that's the only such key on the host. you can't send values to items key[1] and key[2], because they don't exist. there is only one item here. as such, there isn't any functional benefit from being able to send to an item by resolved usermacros.

also, imagine there being an item key[1] and $MACRO resolving to 1 (possible and valid configuration) - which item should receive such value then ?

thus item is uniquely identified by it's key and key alone. and full key, and exactly as specified in the item config.

as for polling, that's different - you want to be able to customise some parameter per host, but that still gets you one item only, just the _way_ you poll it is changed. the way you obtain the value in the case of zabbix sender is not limited or controlled by the item key in any way, thus you only have to uniquely identify an item you want to send the data to.

i feel a desire to explain this in person, waving my hands frantically and drawing on a whiteboard, so i can see that it may be confusing at first

[Spectroman]

Documentation Failure?

Yes, I also agree with those points described.

Although you should agree that, if you have a User Macro, with a certain value, the zabbix sender, should be recognized as using a key that later can be verified through a later substitution, as in:

1) host profile user macro: {$VALUE} => NAME1
1.a) key name = keyname[{$VALUE}]
1.b) zabbix_sender = -k "keyname[NAME1]

2) host profile user macro: {$VALUE} => NAME2
2.a) key name = keyname[{$VALUE}]
2.b) zabbix_sender = -k "keyname[NAME2]

In my mind, this should be valid, in the same hand that when you refer to:

3) host profile user macro: no such macros
3.a) key name = keyname[{$VALUE}]
3.b) zabbix_sender = -k "keyname[NAME3]

**shall not be validated**

-----

Better then, if just make an update on the documentation to except zabbix_sender to use such feature, as he is not "translating backwardly" the user macro value (not him, but zabbix and some of its internal functions) -- maybe add this to a TODO list?

Thanks for the healthy discussion anyway ;-)

Regards,

[richlv]

hrm. i believe reverse-resolving usermacros for zabbix sender data would be confusing and inconsistent (it would at least leave a perception of not being consistent).

docs could always be improved, though - if you look at 2.0 manual (where most work is happening now), do you see a location where you'd like to have a tip or example that's missing ?