Database growth after 2.2.1 Update

[aderr]

Recently, I have been running low on disk space on my zabbix server, running 2.2.1 on CentOS 6, MySQL backend, 51.05 new values/sec.

After investigating, I noticed that the DB was growing larger than I had anticipated, and looking at the history, it all started the day that I upgraded to Zabbix 2.2.1.



You can see in the image that the upgrade was completed on 11/19/2013. Before that date, the free space on / was at a constant ~34%.

I already have plans to clean up the database (new housekeeper settings and MySQL optimization), but I was just curious if anyone else has noticed this same behavior since upgrading.

Since I upgraded, nothing else has changed on the server, in fact I have removed some hosts since then.

Thanks!

[jhenry]

Have you looked at the new housekeeper settings in 2.2? It's not controlled by the server config file anymore. Go to Administration -> General and then change the dropdown at top right to Housekeeping. You'll probably find that housekeeping is disabled or running less frequently than before and that's why the DB is growing.

[aderr]

Yup, the housekeeper is enabled in the GUI and server config and set to only keep 90 days of data in the history and 180 in trends.

I can confirm that the housekeeper is running, every hour it is deleting up to the maximum number of items allowed as set by the server config file (10000).

Thanks.

[sles]

I'd like to confirm this- database grows every day, don't know from 2.2.0 or 2.2.1...

[aderr]

Thanks for the confirmation. Do you see a similar pattern of a steady size and then growth happening after an upgrade?

I'd like to confirm this- database grows every day, don't know from 2.2.0 or 2.2.1...

[rcollier]

I too have noticed that the mysql database in my environment is growing faster after upgrading to 2.2.1. Housekeeping has been enabled in the GUI just like your scenario.

[aderr]

I too have noticed that the mysql database in my environment is growing faster after upgrading to 2.2.1. Housekeeping has been enabled in the GUI just like your scenario.

Roughly, how large is your database, aka ibdata1 file? Mine is around 20GB currently.

Thanks.

[rcollier]

The size of the ibdata1 file in my environment is roughly 5GB. My environment is rather small, at the moment I am monitoring 17 hosts with 24.53 values per second.

[Tim_0200]

Was a cause/solution ever discovered?

[rcollier]

Yes, at least in my case there was. I discovered that my housekeeping wasn't running. All housekeeping is managed through the UI now. Once I configured housekeeping, the disk usage went back to normal levels (I even gained a little back).

[Tim_0200]

Thanks for your reply. I actually JUST created another post trying to figure out how the process works...


How long did it take for the housekeeper to clean everything up? Was it immediate? Did the housekeeper run longer than normal the first time? How big was/is your database? How many items was the housekeeper deleting from the history table?

[rcollier]

You'll have to forgive me because I don't remember any specifics (this happened a few months ago). I do remember that the housekeeper took longer to run because it had to clean out a lot of data. The housekeeper would run as long as 2 minutes in some cases. Now that it has caught up, it runs in about 10 seconds.

Edit: It took a few weeks for it to normalize.

[Tim_0200]

But it essentially got rid of all of the old data in the first instance it ran? In my case, it hasn't.

[rcollier]

No, it didn't. It took about a week or two for it to finally normalize. How long are you holding onto data? I have my history parameter set to 14 days.

[Tim_0200]

No, it didn't. It took about a week or two for it to finally normalize.

Thanks!!! That helps me out a lot.