Zabbix performance optimizing

[avecsi]

Hi All,

Starting Zabbix Server. Zabbix 2.0.2 (revision
****** Enabled features ******
SNMP monitoring: YES
IPMI monitoring: YES
WEB monitoring: YES
Jabber notifications: YES
Ez Texting notifications: YES
ODBC: NO
SSH2 support: YES
IPv6 support: YES
******************************
server #1 started [configuration syncer #1]
server #2 started [db watchdog #1]
server #3 started [poller #1]
server #4 started [poller #2]
server #5 started [poller #3]
server #7 started [poller #5]
server #9 started [trapper #1]
server #6 started [poller #4]
server #10 started [trapper #2]
server #11 started [trapper #3]
server #12 started [trapper #4]
server #13 started [trapper #5]
server #14 started [icmp pinger #1]
server #8 started [unreachable poller #1]
server #16 started [housekeeper #1]
executing housekeeper
server #15 started [alerter #1]
server #17 started [timer #1]
server #18 started [http poller #1]
server #20 started [history syncer #1]
server #21 started [history syncer #2]
server #19 started [discoverer #1]
server #22 started [history syncer #3]
server #23 started [history syncer #4]
server #24 started [escalator #1]
server #25 started [proxy poller #1]
server #0 started [main process]
server #26 started [self-monitoring #1]







How can I avoid that big Zabbix queue ?

I use 3 box for zabbix (db,z-server,webinterface). The z-server load is under 0.1

Thanks,
Andrew

[tchjts1]

Zabbix 2.0.2? Hmmm, I think it would benefit you to move up to 2.0.9 or higher in the 2.0.x series.

As far as optimizing, please see the last part of this post:

https://www.zabbix.com/forum/showthread.php?t=41219

[avecsi]

Hi tchjts1,

Yesterday I changed on the pollers:
StartPollers=10
StartIPMIPollers=0
StartPollersUnreachable=1
StartTrappers=10
StartPingers=1
StartDiscoverers=0
StartHTTPPollers=5

Default was:
StartPollers=5
StartIPMIPollers=0
StartPollersUnreachable=1
StartTrappers=5
StartPingers=1
StartDiscoverers=1
StartHTTPPollers=1

Housekeeper is still running for 4 hours and I cannot increase the "values processed by zabbix server per second".

Do you know who can I increase that value?

Thanks,

[pc99096]

1. check your dashboard - "Required server performance, new values per second"

2. check your queue - Administration -> Queue

3. check zabbix_server.conf:

CacheSize=64M
StartDBSyncers=10
HistoryCacheSize=16M
TrendCacheSize=8M
ValueCacheSize=48M

don't take my values as a reference

[tchjts1]

What you really need to do is re-read the last paragraph of the link I gave in my above post, then provide a screenshot of your graphs regarding how Zabbix internal processes are be utilized. That is the only way to tune your specific setup and if you show them to us with a 1 day view, we can help.

And please do not set your StartDBSyncers any higher than the default setting of 4. The only time this should be adjusted higher is in cases where many thousands of hosts are being monitored.

[avecsi]

Hi tchjts1,

My images was removed where I uploaded them.
Now you should see them in my first post.
The graph was taken before I changed the pollers number.

To change the shared memory I should first change the shmmax in my kernel and based on the graph I have enough free cache.

Number of hosts (monitored/not monitored/templates) 303
163 / 63 / 77
Number of items (monitored/disabled/not supported) 10509
7072 / 3190 / 247
Number of triggers (enabled/disabled)[problem/unknown/ok] 3209
3137 / 72 [62 / 0 / 3075]
Required server performance, new values per second 213.17

Thanks,
Andrew

[avecsi]

Here are the new graphs with the changed configuration:






Zabbix queue is high because I started to use maintenance period with no data collection on new hosts what is a (known&fixed)bug in zabbix.
(You have to educe 200 item from the graph)

[tchjts1]

Check your housekeeper settings on your Zabbix server in zabbix_server.conf
I think these settings would work for you:

### Option: HousekeepingFrequency
#       How often Zabbix will perform housekeeping procedure (in hours).
#       Housekeeping is removing unnecessary information from history, alert, and alarms tables.
#
# Mandatory: no
# Range: 1-24
# Default:
# HousekeepingFrequency=1


### Option: MaxHousekeeperDelete
#       The table "housekeeper" contains "tasks" for housekeeping procedure in the format:
#       [housekeeperid], [tablename], [field], [value].
#       No more than 'MaxHousekeeperDelete' rows (corresponding to [tablename], [field], [value])
#       will be deleted per one task in one housekeeping cycle.
#       SQLite3 does not use this parameter, deletes all corresponding rows without a limit.
#       If set to 0 then no limit is used at all. In this case you must know what you are doing!
#
# Mandatory: no
# Range: 0-1000000
# Default:
# MaxHousekeeperDelete=500
 MaxHousekeeperDelete=400

And also, I think you could increase your StartTrappers setting.
I monitor around 600 hosts and have an NVPS of 936. This is my Trapper setting. You probably don't need this many, though.

### Option: StartTrappers
#       Number of pre-forked instances of trappers.
#
# Mandatory: no
# Range: 0-1000
# Default:
# StartTrappers=5
StartTrappers=60

And you may also wish to bump your configuration cache setting up a little bit. Mine is set at this: (Again for the number of hosts and NVPS I have)

### Option: CacheSize
#       Size of configuration cache, in bytes.
#       Shared memory size for storing host, item and trigger data.
#
# Mandatory: no
# Range: 128K-1G
# Default:
# CacheSize=8M
CacheSize=254M

After any changes to zabbix_server.conf, you need to restart the zabbix_server process. You do not need to do a server reboot.

[avecsi]

what is NVPS ?

[aib]

Required server performance, new values per second 213.17

NVPS = new values per second

[tchjts1]

what is NVPS ?

See below screenshot.

[avecsi]

Thanks All.

I did the following changes on the my zabbix_server.conf:
MaxHousekeeperDelete=200
StartTrappers=15
CacheSize=16M
HistoryCacheSize=16M
HistoryTextCacheSize=32M
StartPollers=10
StartIPMIPollers=0
StartPollersUnreachable=1
StartTrappers=10
StartPingers=1
StartDiscoverers=1
StartHTTPPollers=5

Ubuntu kernel change:
kernel.shmmax=67108864
kernel.shmall=4194304

I will update the topic with the new graps after 24hour

[avecsi]

With new settings :





My Housekeeper still running 3-4hours

[Ricarzd]

Hi,

I had a similar problem and even sometimes that occasionally I have some error. You can:

Check your zabbix configuration
Check mysql configuration, i can help you
Check ulimit for your SO

[avecsi]

Hi,

I had a similar problem and even sometimes that occasionally I have some error. You can:

Check your zabbix configuration
Check mysql configuration, i can help you
Check ulimit for your SO

Hi,

what should I check in my zabbix configuration?
I use postgresql
What is ulimit?

ulimit -Sa
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 63751
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 100
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited