Scheduled Housekeeping

[bashman]

I have no proxy. Should i get some? Would it solve the problem with performance?

I think using proxies should improve zabbix db performance.

[bashman]

We have a smaller MySQL database (6 GB).
Our current configuration is:
Number of hosts (monitored/not monitored/templates) 272 112 / 10 / 150
Number of items (monitored/disabled/not supported) 7012 5082 / 1930 / 0
Number of triggers (enabled/disabled)[true/unknown/false] 2343 2294 / 49 [3 / 109 / 2182]
Required server performance, new values per second 19.7395

We have left default housekeeper configuration : it runs every hour.
We haven't observed any slowdown.
Maybe you can try running it more than once a day.

Regards,
Alixen

Which is your polling interval?

[MrKen]

Zabbix Housekeeper works just fine!

#1 Zabbix Server 1.6.5
Database size 26G
Hosts monitored 167
Items monitored 9883
Required Server Performance 109.0018 values/sec

12463:20100507:140156 Executing housekeeper
12463:20100507:140248 Deleted 227572 records from history and trends
[That's 52 seconds]

***********************************************

#2 Zabbix Server 1.6.6
Database size 7.1G
Hosts monitored 36
Items monitored 2376
Required Server Performance 20.3117 values/sec

21092:20100507:131441 Executing housekeeper
21092:20100507:131449 Deleted 68848 records from history and trends
[That's 8 seconds]

***********************************************

I must admit that it wasn't always like this. Housekeeper used to take around an hour, and the frontend was slower than a wet weekend. I used to believe that a cpu load of 2.5 was normal, now it's purring along at around 0.6 on the first box and 0.2 on the second box.
So, what changed? Tuning my.cnf and reducing the number of StartPollers and Trappers. Start with tuning my.cnf. It takes time and lots of trial and error.

MrKen

[bashman]

I must admit that it wasn't always like this. Housekeeper used to take around an hour, and the frontend was slower than a wet weekend. I used to believe that a cpu load of 2.5 was normal, now it's purring along at around 0.6 on the first box and 0.2 on the second box.
So, what changed? Tuning my.cnf and reducing the number of StartPollers and Trappers. Start with tuning my.cnf. It takes time and lots of trial and error.

MrKen

I have tuned my.cnf already, but I think I'm missing something in my zabbix_server.conf:

############ ADVANCED PARAMETERS ################

### Option: StartPollers
#       Number of pre-forked instances of pollers.
#       You shouldn't run more than 30 pollers normally.
#
# Mandatory: no
# Range: 0-255
# Default:
StartPollers=100

### Option: StartIPMIPollers
#       Number of pre-forked instances of IPMI pollers.
#
# Mandatory: no
# Range: 0-255
# Default:
StartIPMIPollers=5

### Option: StartPollersUnreachable
#       Number of pre-forked instances of pollers for unreachable hosts.
#
# Mandatory: no
# Range: 0-255
# Default:
StartPollersUnreachable=10

### Option: StartTrappers
#       Number of pre-forked instances of trappers
#
# Mandatory: no
# Range: 0-255
# Default:
StartTrappers=100

### Option: StartPingers
#       Number of pre-forked instances of ICMP pingers.
#
# Mandatory: no
# Range: 0-255
# Default:
StartPingers=50

### Option: StartDiscoverers
#       Number of pre-forked instances of discoverers.
#
# Mandatory: no
# Range: 0-255
# Default:
StartDiscoverers=1

### Option: StartHTTPPollers
#       Number of pre-forked instances of HTTP pollers.
#
# Mandatory: no
# Range: 0-255
# Default:
StartHTTPPollers=1

### Option: ListenIP
#       Listen interface for trapper.
#       Trapper will listen on all network interfaces if this parameter is missing.
#
# Mandatory: no
# Default:
#ListenIP=172.17.64.123


### Option: HousekeepingFrequency
#       How often Zabbix will perform housekeeping procedure (in hours).
#       Housekeeping is removing unnecessary information from history, alert, and alarms tables.
#       If PostgreSQL is used, suggested value is 24, as it performs VACUUM.
#
# Mandatory: no
# Range: 1-24
# Default:
# HousekeepingFrequency=1

### Option: DisableHousekeeping
#       If set to 1, disables housekeeping.
#
# Mandatory: no
# Range: 0-1
# Default:
# DisableHousekeeping=0

### Option: SenderFrequency
#       How often Zabbix will try to send unsent alerts (in seconds).
#
# Mandatory: no
# Range: 5-3600
# Default:
# SenderFrequency=30

### Option: CacheSize
#       Size of configuration cache, in bytes.
#       Shared memory size for storing hosts and items data.
#
# Mandatory: no
# Range: 128K-1G
# Default:
# CacheSize=8M

### Option: CacheUpdateFrequency
#       How often Zabbix will perform update of configuration cache, in seconds.
#
# Mandatory: no
# Range: 1-3600
# Default:
# CacheUpdateFrequency=60

### Option: HistoryCacheSize
#       Size of history cache, in bytes.
#       Shared memory size for storing history data.
#
# Mandatory: no
# Range: 128K-1G
# Default:
# HistoryCacheSize=8M

### Option: TrendCacheSize
#       Size of trend cache, in bytes.
#       Shared memory size for storing trends data.
#
# Mandatory: no
# Range: 128K-1G
# Default:
# TrendCacheSize=4M

### Option: HistoryTextCacheSize
#       Size of text history cache, in bytes.
#       Shared memory size for storing character, text or log history data.
#
# Mandatory: no
# Range: 128K-1G
# Default:
# HistoryTextCacheSize=16M

### Option: NodeNoEvents
#       If set to '1' local events won't be sent to master node.
#
# Mandatory: no
# Range: 0-1
# Default:
# NodeNoEvents=0

### Option: NodeNoHistory
#       If set to '1' local history won't be sent to master node.
#
# Mandatory: no
# Range: 0-1
# Default:
# NodeNoHistory=0

### Option: Timeout
#       Specifies how long we wait for agent, SNMP device or external check (in seconds).
#
# Mandatory: no
# Range: 1-30
# Default:
Timeout=3

### Option: TrapperTimeout
#       Specifies how many seconds trapper may spend processing new data.
#
# Mandatory: no
# Range: 1-300
# Default:
TrapperTimeout=300

### Option: UnreachablePeriod
#       After how many seconds of unreachability treat a host as unavailable
#
# Mandatory: no
# Range: 1-3600
# Default:
# UnreachablePeriod=45

### Option: UnavailableDelay
#       How often host is checked for availability during the unavailability period.
#
# Mandatory: no
# Range: 1-3600
# Default:
# UnavailableDelay=60

### Option: UnreachableDelay
#       How often host is checked for availability during the unreachability period
#
# Mandatory: no
# Range: 1-3600
# Default:
# UnreachableDelay=15

### Option: AlertScriptsPath
#       Location of custom alert scripts
#
# Mandatory: no
# Default:
AlertScriptsPath=/etc/zabbix/alert.d/

### Option: ExternalScripts
#       Location of external scripts
#
# Mandatory: no
# Default:
ExternalScripts=/etc/zabbix/externalscripts

### Option: FpingLocation
#       Location of fping.
#       Make sure that fping binary has root ownership and SUID flag set!
#
# Mandatory: no
# Default:
# FpingLocation=/usr/sbin/fping

### Option: Fping6Location
#       Location of fping6.
#       Make sure that fping binary has root ownership and SUID flag set
#
# Mandatory: no
# Default:
# Fping6Location=/usr/sbin/fping6

### Option: SSHKeyLocation
#       Location of public keys for SSH checks
#
# Mandatory: no
# Default:
#SSHKeyLocation=/etc/zabbix/sshKeys
SSHKeyLocation=/home/zabbix/.ssh

### Option: TmpDir
#       Temporary directory.
#
# Mandatory: no
# Default:
TmpDir=/tmp/server

### Option: Include
#       You may include individual files or all files in a directory in the configuration file.
#
# Mandatory: no
# Default:
# Include=

# Include=/etc/zabbix/zabbix_server.general.conf
# Include=/etc/zabbix/zabbix_server/
LogSlowQueries=10000

My log:

4886:20100507:063143.581 Executing housekeeper
4673:20100507:063215.331 Sending list of active checks to [x.x.x.x] failed: host [<hostname>] not found
--
4886:20100507:074255.136 Executing housekeeper
4407:20100507:074301.070 Item [<hostname>:key] error: Get value from agent failed: ZBX_TCP_READ() failed [Interrupted system call]
--
4886:20100507:085142.680 Executing housekeeper
4908:20100507:085213.717 Error doing curl_easy_perform [Server returned nothing (no headers, no data)]

[bashman]

I have no proxy. Should i get some? Would it solve the problem with performance?

Proxy will increase performance of zabbix_server.

Have a look at the documentation: Proxies, Performance tuning.

[Marco Pereira]

Hi,

I have a few questions. what is the relation between "Housekeeper" and "Working Time"? I do need Zabbix to gather information 24/7, and I need to know if I configure Zabbix to work, say from 4 am to 11.59 pm, Housekeeper still works the 24 hours? and when Zabbix starts working again at 4 pm, does it collect data from the proxies, from the period it was not working?

Thank you

[cyber]

Working time is just for visuals... it does not mean that zabbix stops working outside of that time. Data collection is defined in item config, Data collection from proxies is still going on.
Housekeeping is working all the time.