Zabbix 6.2 mySQL - Housekeeper does not appear to work

[WirelessGuru]

Hello,

I have tried everything with housekeeper and adjusting items in the server conf file. I went from 90d history to 30d both adjusting individual items and even override global settings and still the database just continues to grow.

I occasionally get the following message : Zabbix server: Utilization of housekeeper processes over 75%.

I've added more cache memory to the sql server, added more cpu, added more ram, increased pollers. Nothing appears to be able to reduce the db size now sitting almost 500 GB with all tables plus OS.

Events and alerts
Trigger data storage period 30d
Service data storage period 1d
Internal data storage period 1d
Network discovery data storage period 1d
Autoregistration data storage period 1d
Services
Enable internal housekeeping checked
Data storage period 30d
User session
Enable internal housekeeping checked
data storage period 30d
History
Enable internal housekeeping checked
Override item history period checked
Data storage period 30d
Trends
Enable internal housekeeping checked
Override item trend period checked
Data storage period 180d

Anything I can check to see if the housekeeper process is broken?

I'm running Zabbix 6.2 with mySQL.

Thanks,

Dan

[Markku]

Some people say that "OPTIMIZE TABLE xxx" will reduce the MySQL database table size (when data has been deleted), some say it does not help. I don't know, never tried. But yes it is a feature that deleting data from the database does not usually automatically release the disk space. Search the net (or this forum) for shrinking MySQL database size and choose which information and instructions to trust. Remember to have backup before attempting something. Exporting and importing the data with mysqldump to another server will also show you how it affects the size.

Implementing database partitioning (like https://blog.zabbix.com/partitioning...cedures/13531/ ) ensures that old data is efficiently removed from disk, this is what I always do with MariaDB.

Your Zabbix version is quite old and you didn't say if you are using the latest fix version of 6.2, so there can also be some housekeeper bugs.

Markku

[tim.mooney]

Anything I can check to see if the housekeeper process is broken?

Look in your zabbix_server.log. Do you see log entries from each time the housekeeper runs? It should look something like this:

290635:20250720:010344.460 executing housekeeper
290635:20250720:010344.861 housekeeper [deleted <some number here> hist/trends, 0 items/triggers, 0 events, 0 problems, 0 sessions, 0 alarms, 0 audit, 0 autoreg_host, 0 records in 0.400805 sec, idle for 1 hour(s)]

How many times a day is it successfully completing, and roughly how many entries is it deleting every time it runs? We don't need exact numbers for history rows deleted, I'm just trying to get an idea of (number of cleanups/day) x (number of hist/trends deleted per cleanup).

Next, what is your server's new values per second (NVPS)?

[WirelessGuru]

Hi tim.mooney,

Nothing in that log file that that references the housekeeper or key words you listed.

Where would I find NVPS?

Hi Markku,

What is the latest 6.2 fix version? Below is what I find running th ezabbix_server -V command.

zabbix_server (Zabbix) 6.2.1
Revision 2c382976146 25 July 2022, compilation time: Jul 14 2022 11:12:52

Copyright (C) 2022 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.1.1f 31 Mar 2020
Running with OpenSSL 1.1.1f 31 Mar 2020

Would it be easier to upgrade to 7.4 and then do some sort of DB partioning with Timescale DB or would that article from 2021 still be pertinent?

[Markku]

The final 6.2 seems to have been 6.2.9.

If you don't need your existing data, the easiest is to install 7.0 (for long-term support) and TimescaleDB and start from there.

If you need your data and are not comfortable with adding the partitioning yourself, you may want to ask the author of the blog article for paid service to add partitioning (or contact your friendly database admin team if they could help you). Or, you can try to live without partitioning but do some kind of compacting manually now and whenever needed later.

Markku

[tim.mooney]

Nothing in that log file that that references the housekeeper or key words you listed.

Where would I find NVPS?

What setting for DebugLevel do you have in your /etc/zabbix/zabbix_server.conf ? With the default (DebugLevel=3) it should (I think) be logging every time the housekeeper runs. My site is running 6.0.x and every housekeeper run is logged. If your DebugLevel is set at the default and you're still not getting housekeeper logs, you could try running:

sudo zabbix_server -R log_level_increase=housekeeper

and then wait to see what it logs next run. You only really need the informational output from it about what it accomplished and how long it took to run, you don't need debug information from it. If increasing the log level starts to log what looks like debug information from the housekeeper, then you probably want to do 'log_level_decrease=housekeeper' again. We know that housekeeper is running in some capacity because you're getting warnings about it, but we need to know how long it's taking and what it's accomplishing when it does run.

NVPS should either be visible in the "System Information" widget on a dashboard or via the Reports->System Information. You're looking for the "Required server performance, new values per second" information. You may need to be logged in as Admin for it to show up, though.

I would second what Markku says: if you do decide to upgrade, unless you absolutely need the new features in 7.4, you're better off landing on a a long-term support release, which would be 7.0.x. 6.2.x, 6.4.x, 7.2.x, and 7.4.x are for sites that want or need the very latest features and they're OK knowing that the support lifecycle for that release will be shorter than a long-term support release.

[WirelessGuru]

I created a new 7.4 server with Postgres SQL and Timescale DB.

The issue I have not is the export and import of templates, hosts, maps, items, triggers etc.

When I use the GUI, I can only export 50 templates at a time and when I go to import, it says the file is too big.

Then I tried API using ChatGPT for assistance which was able to mass export via API, but I can't seem to import it. Not sure if the syntax from 6.2 to 7.4 is causing an issue. I already figured out the authentication piece which was different.