Problem after DB transfer
Good afternoon.
Sorry for bad English
There was a strange problem and I can't understand why.
Long story short: we have missed Zabbix database growth due to disabled housekeeper to 100+ Gb. The largest table - history and history_uint (55 Gb and 57, respectively).
When it was discovered - in addition, it was found out that mysql server was running without argument innodb_file_per_table, so even if we clean tables OPTIMIZE not free disk space.
It was decided to drag the database to a new server with mysql enabled innodb_file_per_table, which was done directly (mysqldump -uroot -ppwd zabbix | mysql -uroot -ppwd - host = mysql-02 -C zabbix)
Naturally, during database transfer Zabbix server was stopped.
During the night the base moved, in the morning in zabbix_server.conf I changed the path to the database to the new mysql server, and found the problem. Recent events and last history not updating.
Zabbix log contains nothing interesting, just routine - there is a connection to the database, objects are monitored and periodically lose / reconnect, go to the "maintenance" in and out, working housekeepers and clean quietly (range is 1 hour to remove the limit of 3000); objects, triggers, and so on - all over the place, added and removed.
The new mysql server show full processlist; shows the work of housekeepers and rare inserts insert into history (itemid, clock, ns, value) values (30038,1435815998,742992851,4.996824), (31118,1435815998,754607094,81.062772),blah blah blah
The process zabbix_server loads cpu on small percent (0.5% with short peaks up to 20-30%); mysql + about as the load on the drive when working housekeeper 30-50Mb/s, when the housekeeper are not running - 20-50K/s.
It seems to be good, but all the recent events that I see on Zabbix Web Frontend dated yesterday evening (ie, that occurred to stop the Zabbix server). Looking on the graphs - the blank space in the segment last 12 hours (older information showed properly).
If I run the script on the PC which is monitored by Zabbix (sending values through jabber_sender, this script should work force trigger) nothing happens, ie sending work out - but the trigger info (problem) does not appear in Zabbix Web Frontend.
Please tell in what direction to dig, if someone faced a similar problem, or simply knows what is happening.
Thanks in advance.
Sorry for bad English
There was a strange problem and I can't understand why.
Long story short: we have missed Zabbix database growth due to disabled housekeeper to 100+ Gb. The largest table - history and history_uint (55 Gb and 57, respectively).
When it was discovered - in addition, it was found out that mysql server was running without argument innodb_file_per_table, so even if we clean tables OPTIMIZE not free disk space.
It was decided to drag the database to a new server with mysql enabled innodb_file_per_table, which was done directly (mysqldump -uroot -ppwd zabbix | mysql -uroot -ppwd - host = mysql-02 -C zabbix)
Naturally, during database transfer Zabbix server was stopped.
During the night the base moved, in the morning in zabbix_server.conf I changed the path to the database to the new mysql server, and found the problem. Recent events and last history not updating.
Zabbix log contains nothing interesting, just routine - there is a connection to the database, objects are monitored and periodically lose / reconnect, go to the "maintenance" in and out, working housekeepers and clean quietly (range is 1 hour to remove the limit of 3000); objects, triggers, and so on - all over the place, added and removed.
The new mysql server show full processlist; shows the work of housekeepers and rare inserts insert into history (itemid, clock, ns, value) values (30038,1435815998,742992851,4.996824), (31118,1435815998,754607094,81.062772),blah blah blah
The process zabbix_server loads cpu on small percent (0.5% with short peaks up to 20-30%); mysql + about as the load on the drive when working housekeeper 30-50Mb/s, when the housekeeper are not running - 20-50K/s.
It seems to be good, but all the recent events that I see on Zabbix Web Frontend dated yesterday evening (ie, that occurred to stop the Zabbix server). Looking on the graphs - the blank space in the segment last 12 hours (older information showed properly).
If I run the script on the PC which is monitored by Zabbix (sending values through jabber_sender, this script should work force trigger) nothing happens, ie sending work out - but the trigger info (problem) does not appear in Zabbix Web Frontend.
Please tell in what direction to dig, if someone faced a similar problem, or simply knows what is happening.
Thanks in advance.
Comment