Housekeeping: Deleting more at once (hk_period * 4)?

[Linwood]

If I am reading the code correctly, it appears that Housekeeping is limited to removing at most 4 times the house keeping period of data at one time, so if it is set to 1 hour, it will delete at most 4 hours of data, even if the MaxHousekeeperDelete=0.

Is that correct?

So the most I can delete at once by a housekeeping run is to set the period to 24 hours, run it manually, and it will delete 4 days at a time.

Is there a faster approach to this if we want to reduce the amount of data we are keeping (we want to go from 365 to 185 days).

I've been going around in circles why it was not changing until I noticed this (we actually have the period at 1 hour, so it was deleting very little).

Linwood

[Linwood]

Actually it may be worse than that, it appears forcing it to run does not cause another 4 days, there's some kind of time_slept check I'm not quite following.

Is it necessary to recompile and change HK_MAX_DELETE_PERIODS to be a large number? Or is there some better way?

[tim.mooney]

Actually it may be worse than that, it appears forcing it to run does not cause another 4 days, there's some kind of time_slept check I'm not quite following.

Is it necessary to recompile and change HK_MAX_DELETE_PERIODS to be a large number? Or is there some better way?

When I needed to reduce our history, I tried running the housekeeper manually every time it finished, using a script. I got it so that it was deleting stuff about every 3 to 5 minutes, but it was still too slow.

I ended up opening a support ticket (we have paid support with a limited # of cases per year) and they had me rename the existing history and history_uint tables and then create new, blank tables from the renamed tables, and then use SQL queries to copy just X # of days from the renamed table into the empty replacement table. Once I was certain the replacement tables had what I needed, I was able to DROP the renamed tables.

[Linwood]

Well, I like and am comfortable in SQL but as zabbix has gotten more mature and uses caches and such I have gotten more leery about manually changing data. But that might be the simple approach, and more efficient.

I did recompile last night and did some test runs on a test system, and increasing the max delete periods does let it delete a lot more at once, and at present all my clients are running on built-from-source versions, so it is pretty easy to do.

I only recently jumped from 4.5 to 6.0, it looks like partitions have gotten more mainstream in use for fast deletes, it may be well past time I do that in addition. But everything else was working so I just never have (these are fairly small sites, a few thousand hosts is the largest).

I guess I am surprised this is very hard coded in though, almost everything else there's a way to change, at least in a config file if not the gui.

[tim.mooney]

Well, I like and am comfortable in SQL but as zabbix has gotten more mature and uses caches and such I have gotten more leery about manually changing data. But that might be the simple approach, and more efficient.

Agreed. I would have never been willing to do the direct-SQL route if support hadn't suggested it as the best solution for what I was trying to accomplish.