Zabbix Proxy : hystory syncer process 100% busy

[tsalle]

Hello,

We have a large Zabbix Environment, with 5 servers :
- 1 zabbix server
- 1 MySQL Database with partitioning (Housekeeper disabled)
- 1 MySQL Slave Database (for data extraction and reportings)
- 2 proxy

And, since I've reached 500 vps, (~2000 hosts) the server history syncer process is busy at 100% almost all the time.


I've tried to raise the number of DBSyncProcess in server configuration (100 process), but it's still busy (it's better, but not enough)..

Do you have an idea to make zabbix proxy history sync to the master faster ?

Edit :
I'have examined the Mysql "show processlist" results, and show that each insert is very, very long ! (150 seconds).
My history_uint table is partioned by range on clock (1 partition / day) and I have to keep 30 days of history to make manual extracts of raw data at the end of the month.
MySQL is already cusomised (inno_db_buffer_pool_size = 8G, innodb_flush_log_at_trx_commit = 2, ...)
Database Cpu usage I/O wait is ~2% and cpu idle time 97%
So i don't know what is the problem...

Thanks..

[rsvancara]

Zabbix History Syncer Process

Hi,

You might need to post more information for assistance.

1. Zabbix Version
2. Hardware specifications for your database server

[tsalle]

Hi, thanks for your answer. I'm running Zabbix 1.8.10, and zabbix proxies and server are both HS22 (IBM), the database server is a x3550 (2x Intel(R) Xeon(R) CPU E5620 + 12G RAM). It's running MySQL server 5.5.24 with history_uint partitionned.

I think the problem come from zabbix proxy who can't sync with the zabbix server. In the zabbix_proxy database, the number of rows in "proxy_history" table is increasing continuously instead of catching up. When I do a show create table on proxy_history, i have

CREATE TABLE `proxy_history` ( `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT, `itemid` bigint(20) unsigned NOT NULL DEFAULT '0', `clock` int(11) NOT NULL DEFAULT '0', `timestamp` int(11) NOT NULL DEFAULT '0', `source` varchar(64) NOT NULL DEFAULT '', `severity` int(11) NOT NULL DEFAULT '0', `value` text NOT NULL, `logeventid` int(11) NOT NULL DEFAULT '0', PRIMARY KEY (`id`), UNIQUE KEY `id` (`id`), KEY `proxy_history_1` (`clock`) ) ENGINE=InnoDB AUTO_INCREMENT=507058950 DEFAULT CHARSET=latin1

but the nextid in my ids table is greater (nextid = 507081153)

I saw that on older version of zabbix or with postgres database the id whas a INT and sometimes the id fall back to 1.

Is there a way to force Zabbix Proxy send all remaining data to server, or even drop zabbix proxy history and start proxy with no delay ?

Is there a config parameter to increase zabbix proxy history syncers process ?

I've tried to stop zabbix proxy, truncate table proxy_history and do a

ALTER TABLE proxy_history AUTO_INCREMENT=<value of nextid in ids tables +1>

Then restart proxy but after a few minutes the db syncer process come back to 100% busy