Plans for idempotency support in API?

[dennisj]

Hi,
are there any plans to add idempotency support to the API?
What I'm talking about is the ability to create items/hosts/etc. but when they already exist update them instead and return if (any maybe which) changes were made.

I'm looking at this from angle of puppet integration and also have looked at implementing this on the client side but quickly backed off as this would be quite a nightmare to maintain and belongs on the API side anyway where the implementation would be much easier.

Automatisation of host monitoring is a big issue for us and Zabbix doesn't seem to handle this well at all but with idempotency support in the API it would be a *lot* easier to create a skripted solution that automatically creates hosts, assigns templates etc. and keep these values updated when the host config changes.

[Colttt]

maybe you mean AutoDiscovery?

https://www.zabbix.com/documentation/2.0/manual/discovery

[dennisj]

No, discovery works only for extremely simple use-cases and is quite limited in what it can do.

[Colttt]

ok,

sorry my english is not the best, but what would you like to do?and what means idempotency ?

[dennisj]

In a nutshell idempotency means that if you apply a function several times the result does not change. This is an important concept in configuration management system (CMS) like puppet where you define a certain state you want your systems to be in and the CMS tries to keep the system in that state.
For example you want the target system to have a directory '/tmp/test'. Now the configuration system could simply do a 'mkdir /tmp/test' but on the second run this would fail with a 'directory already exists' error i.e. implemented this way the function is not idempotent.
If on the other hand the CMS does use 'mkdir -p /tmp/test' then an existing directory is not considered to be an error condition so now the function is idempotent because it can be called repeatedly and always ends in the same result (the directory '/tmp/test' exists).

What I want to accomplish is to be able to define the monitoring of a host in puppet where all other configuration for the host is specified as well. An example would be:

class { "zabbix-host":
hostname => ["myhost"],
templates => ["apache", "nginx"],
macros => ["apache_port=80", "nginx_host=127.0.0.1", "nginx_port=8080"]
}

class { "zabbix-item":
hostname => ["myhost"],
... (various parameters for the item) ...
}

To accomplish that you would have to write a puppet module for the "zabbix-host", "zabbix-item", etc. classes and this is where the problem appears.

The configuration is applied in regular intervals and the expected behaviour is this:
1) If the configuration is applied for the first time then the new host and item are created.
2) If the configuration is applied again and values haven't changed then no changes should happen in zabbix.
3) If the configuration is applied again and values have changed (e.g. a port number in the example above) then the objects in zabbix should be updated accordingly.

Since the zabbix api does not support idempotency you basically have to implement this on the client side which is a big problem. Think about the item case for example. In order to implement this you have to:
1) Call the api to check if the item already exists in zabbix
2) If the item does not exist you have to create it => done
3) If the item already exists you have to compare alle the parameters of the item in zabbix with the parameters of the item specified in the puppet configuration.
4) If all the parametes are identical then no update is needed => done
5) If parameters have changed then you have to update the item in zabbix accordingly => done.

Now look at the parameter reference for zabbix items:

https://www.zabbix.com/documentation/2.2/manual/api/reference/item/object#item

That is a huge number of parameters and you have to compare each of them individually and that is just for the item objects. You have to do the same for all the other objects in the api. But the worst thing is that you have to keep these checks in sync with changes to the api and make different comparisons dependent on which api version is used on the zabbix server.

In other words implementing this would be a huge amount of work and the result would be exremely fragile and break every time the api changes.

Now imagine if the api would support idempotency. One way to implement this would be to add a "if_exists_update" parameter to the create() call. If this is set to 1 when the user calls create() and the object already exists then the api will automatically turn this call into an update and return the number of changed values.

Let's run through the example above again but this time using the imaginary idempotency-enabled api:
1) Call create('if_exists_update=1') for the item => done

That's it. Also notice that you no longer have to compare all the parameters like above in order to check if the item has changed as this now happens on the api side. This might seem like a simple shifting of work to the api but remember that the api already verifies parameters so it already has to do most of this work anyway which is why implementing it there would be much easier.
But most importantly when the api changes (e.g. a field is added) the client no longer needs to care. The api will now also implicitly compare the value for this new field but the client does not need to care so api changes no longer break this client side code.

Remember that I'm using configuration management/puppet just as an example here. This would make scripting the api much easier in general.

[jan.garaj]

IMHO Zabbix API is not a right place for implementation of idempotency.
Look where is idempotency implemented in example with CMS and file resource '/tmp/test'? In CMS, because mkdir isn't idempotent. I think that CMS is the right place for idempotency. You want to modify mkdir (Zabbix), so I don't agree. But of course maybe I'm wrong, so I need any example and facts
Could you provide me any tool with API where idempotency is used and is useful?

If Zabbix has configuration in files, you don't have this problem, because file resource idempotency is usually implemented in CMS (Puppet, Chef, ...). But Zabbix use Database. I don't see any trivial solution for this case. Some coding is needed and question is where - CMS or Zabbix?

[dennisj]

IMHO Zabbix API is not a right place for implementation of idempotency.
Look where is idempotency implemented in example with CMS and file resource '/tmp/test'? In CMS, because mkdir isn't idempotent. I think that CMS is the right place for idempotency. You want to modify mkdir (Zabbix), so I don't agree. But of course maybe I'm wrong, so I need any example and facts
Could you provide me any tool with API where idempotency is used and is useful?

If Zabbix has configuration in files, you don't have this problem, because file resource idempotency is usually implemented in CMS (Puppet, Chef, ...). But Zabbix use Database. I don't see any trivial solution for this case. Some coding is needed and question is where - CMS or Zabbix?

Did you read my posting above completely? I answer all of your questions in there and provide an example.

The answer as to where to implement this is both sides. You need to implement the resources in the CMS but enhance the api for the idempotency. If you put the idempotency part on the CMS side you essentially destroy the separation of concerns between zabbix api and the CMS which is terrible software design. This also means that the work has to be done multiple times once for each CMS you want to support (puppet, Chef, Salt, Cfengine, Ansible, etc.).
If you enable this in the api then this work can be shared by all CSM implementations, these implemetation become a lot easier to write and stay decoupled from the internals of the zabbix api model.

[jan.garaj]

Better place for your request is a Zabbix Jira.
Please feel free to create feature request for your feature. https://support.zabbix.com/

[xaeth]

if you share the support issue # here I'll vote for it

[r0b]

Hi

I'm aware that this is not a complete answer to the problem you described but I've wrote a module to synchronize part of ansible hosts state with Zabbix. It means that it sets hosts via Zabbix API based on ansible variables:

- groups
- templates
- macros

It's idempotent, so it compares parameters before actually sending changes to API.

I encourage to test, use and comment:

https://galaxy.ansible.com/list#/roles/2110