Hi there, i've a Zabbix 2.4.3 installation.
We rely heavily on zabbix_sender cause data is captured by sensors that produce a log file. We stream that information to zabbix trought the use of zabbix_sender -Ti so we can upload to zabbix the hostit/time/key/data tuple
Due a script looping the same files we discovered that zabbix saves the tuple N times in the DB eventually leading our mysql DB to grow to 16GB. All the tuples was nearly identical.
history_uint tuple:
itemid bigint(20) unsigned
clock int(11) [0]
value bigint(20) unsigned [0]
ns int(11) [0]
I was looking for the best method to avoid this kind of duplicates and i was wondering which layer should be resposabile to check the duplicates.
The first idea was using mysql capabilities to add an UNIQUE index to the tuple:
ALTER TABLE `history_uint` ADD UNIQUE INDEX (`item_id`, `clock`, `value`, `ns`);
This seem to solve the problem but has the side effect of growing zabbix_server.log bigger and bigger cause it has to log every single failed query (not added cause of unique index), but i think i can manage it through syslog. Does zabbix support a way to exclude certain logs from the server log (like duplicate entry?). Is this the right approach to avoid duplicates in history_uint or is there any better?
Thanks in advance
We rely heavily on zabbix_sender cause data is captured by sensors that produce a log file. We stream that information to zabbix trought the use of zabbix_sender -Ti so we can upload to zabbix the hostit/time/key/data tuple
Due a script looping the same files we discovered that zabbix saves the tuple N times in the DB eventually leading our mysql DB to grow to 16GB. All the tuples was nearly identical.
history_uint tuple:
itemid bigint(20) unsigned
clock int(11) [0]
value bigint(20) unsigned [0]
ns int(11) [0]
I was looking for the best method to avoid this kind of duplicates and i was wondering which layer should be resposabile to check the duplicates.
The first idea was using mysql capabilities to add an UNIQUE index to the tuple:
ALTER TABLE `history_uint` ADD UNIQUE INDEX (`item_id`, `clock`, `value`, `ns`);
This seem to solve the problem but has the side effect of growing zabbix_server.log bigger and bigger cause it has to log every single failed query (not added cause of unique index), but i think i can manage it through syslog. Does zabbix support a way to exclude certain logs from the server log (like duplicate entry?). Is this the right approach to avoid duplicates in history_uint or is there any better?
Thanks in advance