Why HouseKeeper doesn't remove data older than 90 days?

[tluo]

I upgraded Zabbix to 2.4 months ago. I didn't realize there was a new tab to enable internal housekeeping until days ago when I found the DB had grown so big. I enabled it, and started to see from zabbix_server.log that every hour the number of history records it removed became non-zero. But when I check the DB, I find there are still a lot of records in the history tables with clock values far before the history storage periods I set. How can I remove them?

I will appreciate any response.

[mulbzh]

hello,

i have he same problem. My database history_uint size is 25,1 Gio



how to clear history ?

[mulbzh]

you can change MaxHousekeeperDelete in zabbix_server configuration, the default is 500 and you can put much more

[tluo]

Mulbzh, thanks. I have tried it, but there are still a lot of old data.

[Linwood]

Try a VERY large number, or zero which is no limit (just be prepared to monitor the database). Depending on the database, the tiny "few at a time" that the default does is very inefficient.

You can even jump start it if you are a bit brave by deleting by hand everything over (say) 100 days, then to see if it's working let zabbix do the remaining and see if it can keep up.

Note after really massive deletes some database maintenance may be appropriate, e.g. vacuum full if postgresql.

[tluo]

Now it is 400000. I find no matter how large it is, each house keeper execution removes 4 hour's data. The zabbix database is still growing slowly.

[Linwood]

Do you have it running only once a day? You can increase (in the server config file) how frequently it runs.

There are numerous criteria it uses, they are documented somewhere, it may be in the server config file itself, I don't recall. I just let mine run every hour and unlimited deletes, and it seems to do no harm for me (your mileage may vary of course).

The real solution appears to be to use partitions for data and drop the partitions instead of deleting the rows, though there are some restrictions there, plus you have to code it. Lots of discussion in various posts on the subject.

[fittim]

Now it is 400000. I find no matter how large it is, each house keeper execution removes 4 hour's data. The zabbix database is still growing slowly.

Can you tell me how you determined how many hours of data it got deleted?

[registration_is_lame]

Now it is 400000. I find no matter how large it is, each house keeper execution removes 4 hour's data. The zabbix database is still growing slowly.

Can you tell me how you determined how many hours of data it got deleted?

[kloczek]

I upgraded Zabbix to 2.4 months ago. I didn't realize there was a new tab to enable internal housekeeping until days ago when I found the DB had grown so big. I enabled it, and started to see from zabbix_server.log that every hour the number of history records it removed became non-zero. But when I check the DB, I find there are still a lot of records in the history tables with clock values far before the history storage periods I set. How can I remove them?

I will appreciate any response.

Stop wasting IOs and start using partitioned history tables.

https://www.zabbix.org/wiki/Docs/howto/mysql_partition

[tluo]

Can you tell me how you determined how many hours of data it got deleted?

I am not sure if I am right, but I use this:

select from_unixtime(clock),value from history where itemid=(select itemid from items where hostid=(select hostid from hosts where name like 'table_name%') and name='CPU $2 time' and key_='system.cpu.util[,idle]') limit 50;

Each time after the housekeeping I can see the time of the first record change 4 hours.