Ad Widget

**ingus.vilnis** · 18-02-2015, 15:11

Hi,

Can you post here the actual zabbix_server.conf file?

Maybe you can try to temporary disable all your monitored hosts ensuring that no new values are being received. Then restart the server and see what happens to ValueCache then.

Best Regards,
Ingus

**Murmelantes** · 18-02-2015, 16:51

Hmm.
Such a shame...

PEBKAC:
I was declaring ValueCacheSize at 2 different places in the conf file, so it was actually still at 8M! :|

I still don't quite understand why this specific network needs more than 8M of Value Cache:
it only has 7 hosts, 824 items, 244 triggers, and requires zabbix treating 11.39 new values per second...

But I now changed it to 16M, and it looks more or less stable at 37% free for the last 30min, and the cache effectiveness really improved.

So I guess I can live with it..

Thanks a lot ingus.vilnis!

**ingus.vilnis** · 18-02-2015, 17:19

Still strange even after you found the duplicated line.

But if so, maybe there are some more misconfigured settings? 37% Value cache free is too bad.

Can you also show graph with Value cache misses?

Best Regards,
Ingus

**Murmelantes** · 18-02-2015, 17:45

zabbix_server.conf looks pretty simple, now:

Code:

grep -Ev '(#.*$)|(^$)' /etc/zabbix/zabbix_server.conf

LogFile=/var/log/zabbix/zabbix_server.log
PidFile=/var/run/zabbix/zabbix_server.pid
DBName=*****
DBUser=*****
DBPassword=*****
SNMPTrapperFile=/var/log/snmptt/snmptt.log
StartSNMPTrapper=1
ValueCacheSize=16M
AlertScriptsPath=/usr/lib/zabbix/alertscripts
ExternalScripts=/usr/lib/zabbix/externalscripts

I'm not sure what could be the cause:
problems in external scripts I'm using, issues on the monitored servers...
No idea!

Regarding the graphs, here is everything I have:
Zabbix has been installed 6 days ago...

The ValueCacheSize has been changed to 16M at 15h08, and I restarted the zabbix-server service 3 times after that:

6009:20150218:150822.683 Starting Zabbix Server. Zabbix 2.4.1 (revision 49643).
11470:20150218:155535.174 Starting Zabbix Server. Zabbix 2.4.1 (revision 49643).
11650:20150218:155605.228 Starting Zabbix Server. Zabbix 2.4.1 (revision 49643).
15839:20150218:162943.585 Starting Zabbix Server. Zabbix 2.4.1 (revision 49643).

So basically, except around those 3 restart, there's almost no "misses" anymore.

Thanks a lot for your help!

**ingus.vilnis** · 19-02-2015, 11:04

Hi,

OMG, 16 Kvps Value cache hits with 7 hosts? Normally you should have 16 vps not 16 000 vps with environment of such size.

In such case set your Value Cache size to 128 or 256 or even 512M. 16M is not enough and your server will crash again as soon as you add another host.

Anyways you must find out why does it happen. Therefore could you please show a screenshot with the triggers and especially trigger expressions you have on your hosts? (blur out confidential details if any)

Best Regards,
Ingus

**Murmelantes** · 19-02-2015, 13:33

I found the culprit: I've been really optimistic setting up my triggers.

I created 2 new triggers to compare the incoming and outgoing traffic on all interfaces to the average daily minimum.

Basically, I was calculating the average minimum for the last week, and triggering an alarm when when the traffic was half this average.

The alarm was cleared when the traffic was back to half the average traffic.

Code:

({TRIGGER.VALUE}=0 &
{Template SNMP Interfaces:ifInOctets[{#SNMPVALUE}].max(10m)}
-(({Template SNMP Interfaces:ifInOctets[{#SNMPVALUE}].min(1d,1d)} + 
{Template SNMP Interfaces:ifInOctets[{#SNMPVALUE}].min(1d,2d)} + 
{Template SNMP Interfaces:ifInOctets[{#SNMPVALUE}].min(1d,3d)} + 
{Template SNMP Interfaces:ifInOctets[{#SNMPVALUE}].min(1d,4d)} + 
{Template SNMP Interfaces:ifInOctets[{#SNMPVALUE}].min(1d,5d)} + 
{Template SNMP Interfaces:ifInOctets[{#SNMPVALUE}].min(1d,6d)} + 
{Template SNMP Interfaces:ifInOctets[{#SNMPVALUE}].min(1d,7d)})/14)<0) |
({TRIGGER.VALUE}=1 &
{Template SNMP Interfaces:ifInOctets[{#SNMPVALUE}].min(10m)}
-(({Template SNMP Interfaces:ifInOctets[{#SNMPVALUE}].avg(7d)})/2)<0)

It was working perfectly, but was too heavy...

It would maybe be possible to recreate a similar behavior through calculated items?
I really liked this alarm...

Anyway, I'm back to more normal values (600 vps and 97% cache free), and I still have some "overly optimistic" triggers that I can disable.
So I now know where the issue was coming from...

Thanks again a lot for your help!

**ingus.vilnis** · 19-02-2015, 13:49

Yes, you were storing basically all collected values for those items for 7 days in the value cache and that is why it failed. Using long time shift functions takes a lot of value cache.

Anyways I am glad you found the problem!

Yes, you can think of calculated items but again be careful to not use time based trigger functions so much that it crashes your value cache.

Or as another option add more RAM. It is said that a fast Zabbix setup is where whole DB fits into RAM so you still have space for improvement.

Best Regards,
Ingus

Ad Widget

Zabbix 2.4.1 "Value Cache is Fully used"

Zabbix 2.4.1 "Value Cache is Fully used"

Comment

Comment

Comment

Comment

Comment

Comment

Comment