Zabbix value cache working in low memory mode

[washington]

Hi i see this messenger in my dashboard:

Zabbix value cache working in low memory mode

but i already change the file /etc/zabbix/zabbix_server.conf:

### Option: VMwareCacheSize
# Size of VMware cache, in bytes.
# Shared memory size for storing VMware data.
# Only used if VMware collectors are started.
#
# Mandatory: no
# Range: 256K-2G
# Default:
#VMwareCacheSize=8MB (valor padrado)
VMwareCacheSize=2G

### Option: CacheSize
# Size of configuration cache, in bytes.
# Shared memory size for storing host, item and trigger data.
#
# Mandatory: no
# Range: 128K-8G
# Default:
CacheSize=16G

### Option: HistoryIndexCacheSize
# Size of history index cache, in bytes.
# Shared memory size for indexing history cache.
#
# Mandatory: no
# Range: 128K-2G
# Default:
# HistoryIndexCacheSize=4M
HistoryIndexCacheSize=200M

### Option: ValueCacheSize
# Size of history value cache, in bytes.
# Shared memory size for caching item history data requests.
# Setting to 0 disables value cache.
#
# Mandatory: no
# Range: 0,128K-64G
# Default:
# ValueCacheSize=8M
ValueCacheSize=100M


What can i do to resolve this problem?
Thanks

[washington]

I always change the values and so zabbix comes back work normaly, but i need to know what is happen, some body help me please.

[youngy]

Hi,

We have been seeing the same message from time to time. So far I have just been bumping the value up and restarting.

This thread was one of the first I found on the subject. So thought I would post here as well.

The last time this happened though I managed to find an historical graph for this value and I see it going from roughly 80% free to warning levels in slightly less than 24 hours.

After averaging 80% free for months.

So like Washington above I too would like to know why this is happening.

[ingus.vilnis]

In short Value Cache is used for storing item values for evaluating trigger expressions and calculated items.

Say you have a trigger which evaluates the average values for the last 1 hour (trigger function avg(1h)=...) then all item values for this hour are put into value cache for faster calculations. Otherwise they would be acquired from the database every time the trigger needs to be evaluated and that happens upon arrival of every new item value.

And now imagine someone changed or added such trigger to calculate average for a day, or even worse a week. All those values go into value cache and it drops easily as every new item value for the trigger goes there.

Zabbix cache usage graph on Zabbix server (from the default template) is a good place where to see how full the caches are. But they will not tell exactly what is filling up the caches though.

[youngy]

Thanks for the reply.

I was looking at the graphs for the last month or so and generally speaking the cache is (on average) 80% free. I do have some triggers using average values. But I should be back to filling up the cache post restart if all things are equal.

And this has not happened so far.

It's somewhat difficult to find what ate all of the cache when it happens by the look of it.

[youngy]

Thanks for the post Charlie

Currently my cache is set as follows:

CacheSize=2048M

My environment is not growing. And the cache has been (on average) 76.02% free since my last occurrence of this issue.

I would also say the free% was basically the same until - for some reason - I went into low mode within a 24 hour period.

Next time this occurs I'll be paying careful attention to the type(s) of triggers at play at the time.

[youngy]

So for this round I see the cache get used for a single 24 hours period

last min avg max
zabbix value cache, % free [all] 80.54% 8.86% 42.28% 90.39%
Trigger: Less than 5% free in the value cache [<5]

During that time above I go into low memory mode. I restart earlier this morning and of course see:

Zabbix value cache operating mode (zabbix:zabbix[vcache,cache,mode]): Normal (0)

So back to normal. And interestingly back to the same level in percentage terms free.

Even though I have now bumped:

CacheSize=4096M

Half the maximum value that can be set.

Am I adjusting the right value in the conf file for the trigger?

[ingus.vilnis]

You have to adjust the ValueCacheSize parameter. Set it to e.g. 1G.
I believe you can decrease the CacheSize as well.

In general you want to keep all caches in 70-100% free range. But it is also not the best idea to allocate max of possible configurable memory to them. Keep an eye on the Zabbix cache usage graph for a day, if you wish post it here, list current values of all caches in the config file and then some further suggestions can be made.

[sjohn2024]

i am having this issue when i run command:
systemctl restart zabbix-server
getting the following error:
job for zabbix-server.service failed becuae the control process exited with error code.
see "systemcrl status zabbix-server.service" and "journalctl -xeu zabbix-server.service" for details.

any idea?

[youngy]

Thanks Ingus

What you have said makes sense and I'll do both the next time I see this happen. Actually I have set now - just waiting for a restart window.

Cheers

[youngy]

Hi Ingus,
Just letting you know that I have made those changes and so far things have remained stable and I have not seen the issue occur again.
Hopefully this may help someone else in the future.
Thanks

[ingus.vilnis]

Hi,

Great! Glad you got this resolved.

[avidec87]

Hi Everyone,

I also got the same error in my new Zabbix environment using 4.0.

I have 265653 items and 202698 trigger running on the the server by default setting for ValueCacheSize property in 8M.

I have changed the value from 8M to 100M and restarted the server.

For now issue seems to be resolved but I'll keep looking into for few days and if everything is fine than great else I'll provide update for resolution.

[amandavernon]

Hi Everyone,

I also got the same error in my new Zabbix environment using 4.0.

I have 265653 items and 202698 trigger running on the the server by default setting for ValueCacheSize property in 8M.

I have changed the value from 8M to 100M and restarted the server.

For now issue seems to be resolved but I'll keep looking into for few days and if everything is fine than great else I'll provide update for resolution.

Thank you for the solution. I am experiencing the same issue and home changing to 100M will help in my case too.
buy essays cheap

[avidec87]

Yes, IT should though depend upon your requirement. Please change the value, keep it monitoring and increase it, if requires.

[nileshkahar]

I did "# systemctl restart zabbix-server" & all back to normal.

[azer1020]

Any permanent solution ? i tried different method but always got "Zabbix value cache working in low memory mode" message.