disk space needed - housekeeper hasn't run in ages

[Sebneu]

Hi everyone

I desperately need disk space on my server running zabbix. Now I realized the housekeeper hasn't running for ages.
There are values in trend/history which are over one year old though the value of that item is set to 30/365 respectively.
So I realized I need a big cleaning session.
Is there any way to do this? Activating the housekeeper only cleans 4 days of old data per day (or 4 hours per hour). This is far to slow for me, meaning I have to wait for 3 months to get rid of one years obsolete data. Is there any way to increase this?

I know that I need a VACUUM FULL in pgsql later on and I cleand up my items aswell (reduction of interval and history time).

Kind regards
Sebneu

[harmonica]

Hi,

What´s your Zabbix version? If 2.2.X from official documentation:

Housekeeper changes

The DisableHousekeeping server configuration option is supported no more. Instead, finer controls are located in the frontend, in Administration → General → Housekeeper, allowing to selectively enable/disable housekeeping processes for specific tables.

[Sebneu]

harmonica, thanks for your reply

I've activated those settings yesterday. But the housekeeping only deletes about 100,000 entries per run. But I guess there are about 80,000,000 entries which could be deleted (this is only in history_uint). This is still too slow

Regards
Sebneu

[jan.garaj]

Push your zabbix server housekeeper settings to the maximum :-):

MaxHousekeeperDelete=1000000
HousekeepingFrequency=24

=>24M deleted records per day

But keep eyes on your iowait CPU time and disk IOps metrics.

https://www.zabbix.com/documentation/2.4/manual/appendix/config/zabbix_server

[Sebneu]

jan.garaj, thank you for the hint. I'll have a go.

I already retrieved some disk space by some sql hacking,

DELETE FROM history_uint WHERE age(to_timestamp(history_uint.clock)) > interval '180 days' ;

but I guess your way is safer ;-)

Regards
Sebastian

[Sebneu]

Hi jan.garaj

Push your zabbix server housekeeper settings to the maximum :-):

MaxHousekeeperDelete=1000000
HousekeepingFrequency=24

=>24M deleted records per day

This didn't really work - maybe because I am still using postgresql 8.4 or zabbix 2.2. Or maybe the second part of the zabbix manual entry for HousekeepingFrequency overruled MaxHousekeeperDelete.

Note: To prevent housekeeper from being overloaded (for example, when history and trend periods are greatly reduced), no more than 4 times HousekeepingFrequency hours of outdated information are deleted in one housekeeping cycle, for each item. Thus, if HousekeepingFrequency is 1, no more than 4 hours of outdated information (starting from the oldest entry) will be deleted per cycle.

I sticked to deleting old data manually, followed by a vacuum and a reindex.

My worst tables were history_uint and history. Here is what I did to clean up history_uint

DELETE FROM history_uint WHERE age(to_timestamp(clock)) > interval '40 days';
VACUUM FULL ANALYSE history_uint;
REINDEX INDEX history_uint;

Now I have got 20G of free disk space - this is far too much

Thanks for your help anyway
Sebneu

[tvtue]

Hi,
I have mostly the same problem. I forgot to put the right settings in zabbix_server.conf when updating to 2.2.2. My history_uint table is way too big and housekeeper is too slow to clear all old entries.

Was your command save as of what DB consistency is concerned?
Then I would run that too.

There is that table "housekeeper" which is related to what the housekeeper is doing and if one manually deletes lines from the tables history_* and trends_* this table should be updated too, shouldn't it ?

Thanks and regards,
Timo

[Sebneu]

Hi

I haven't had any problems yet. So I guess it's safe.

There is that table "housekeeper" which is related to what the housekeeper is doing and if one manually deletes lines from the tables history_* and trends_* this table should be updated too, shouldn't it ?
Timo

I am not sure about it, but it sounds like a good idea. But I don't know anything about it....

Regards
Sebneu