Zabbix-proxy and data arrival delay ( more than 2 hrs delay)

[zabbixfk]

Hello,

I am seeing systems which are monitered by zabbix proxy are not updating data, more than 2 hrs of delay to zabbix server.

zabbix proxy is running on a vm with 4G ram, 4core processor.
zabbix master is running on 32G ram, with 8core processor.

Is there anything specific things to be changed, pls advice

Nodes which are mixture of snmp/zabbix agent items which are monitored by zabbix proxy are close to 270+.

zabbix_proxy conf

Server=172.17.1.108
Hostname=NM-Zabbix-Proxy
LogFile=/var/log/zabbix/zabbix_proxy.log
DebugLevel=4
PidFile=/var/run/zabbix_proxy.pid
DBName=zabbix
DBUser=zabbix
DBPassword=zabbix
ConfigFrequency=600
DataSenderFrequency=5
StartPollers=30
StartPollersUnreachable=30
StartTrappers=30
StartPingers=30
StartSNMPTrapper=1
ListenIP=192.168.1.67
CacheSize=512M
HistoryCacheSize=512M
HistoryTextCacheSize=256M
Timeout=30
UnreachablePeriod=60
FpingLocation=/usr/local/sbin/fping
LogSlowQueries=300000

zabbix queue


zabbix server conf

LogFile=/var/log/zabbix/zabbix_server.log
LogFileSize=50
DebugLevel=2
PidFile=/var/run/zabbix/zabbix_server.pid
DBName=zabbix
DBUser=zabbix
DBPassword=zabbix
StartPollers=45
StartPollersUnreachable=12
StartTrappers=45
StartPingers=45
StartDiscoverers=3
StartSNMPTrapper=1
ListenIP=0.0.0.0
HousekeepingFrequency=1
MaxHousekeeperDelete=500
SenderFrequency=300
CacheSize=1G
CacheUpdateFrequency=300
StartDBSyncers=4
HistoryCacheSize=128M
TrendCacheSize=1G
HistoryTextCacheSize=128M
Timeout=30
TrapperTimeout=120
UnreachablePeriod=600
UnavailableDelay=120
AlertScriptsPath=/etc/zabbix/alert.d/
FpingLocation=/usr/local/sbin/fping
LogSlowQueries=1

mysql_conf

datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
symbolic-links=0
slow_query_log_file=/var/log/mysql/slow-query.log
long_query_time = 10
log-queries-not-using-indexes=YES
innodb_lock_wait_timeout=500
innodb_locks_unsafe_for_binlog=1
expire_logs_days=5
max_binlog_size=100M
innodb_buffer_pool_size=20G
innodb_file_per_table
max_allowed_packet = 64M
innodb_additional_mem_pool_size = 30M
innodb_thread_concurrency = 8
key_buffer_size = 60M
max_connections=290
table_cache=4096
query_cache_size = 96M
tmp_table_size = 32M
thread_cache_size = 64
sort_buffer_size = 12M
query_cache_limit=64M
thread_cache_size=512
wait_timeout=300
read_buffer_size=2M
read_rnd_buffer_size=8M
join_buffer_size=8M
log-bin=/var/log/mysql/mysql-bin.log
binlog-do-db=zabbix
server-id=1
[mysqld_safe]
log-error=/var/log/mysql/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

zbx_server queue


ANy pointers are greatly helpful.

Thanks

[Pada]

What does your Zabbix dashboard say for "Required server performance, new values per second" ?

I would recommend that you install the Zabbix_Server and Zabbix_Proxy templates, so that you can determine what percentages of the pollers and other Zabbix processes are busy. Unfortunately the Zabbix Proxy monitoring is only available from version 2.2

If you are monitoring many hosts (or items), then I would highly recommend that you use Zabbix Agent (active) and not the passive ("Zabbix Agent") item. Active items uses a lot less resources on your proxy/server.
From v2.2.3 the Proxies & Servers can support bulk SNMP requests, where they query multiple SNMP items with a single request. See https://www.zabbix.com/documentation...itemtypes/snmp for more detail.

You may want to decrease the timeout values, because 30 seconds is an awful lot for the SNMP Timeout.

If your hosts that make use of SNMP does not return values immediately, then you should have a look at it! Like we had 1 or 2 SNMP items in our setup that would take 3-5 seconds to return, which cause lots of issues for us, because with a timeout of 3 seconds it caused the whole host to go into a unavailable kind of state.

So we basically got rid of all our passive kind of monitoring items and replaced them with Zabbix Agent (active) or Zabbix Trapper items. Our Zabbix proxy instances are like EC2 Micro instances, and they do roughly 100 hosts / 2000 items (75 new values per second).
Decreasing the Update Interval an also have a dramatic effect. Like I won't recommended to use anything lower than 30s, unless you have to for only a few items.

[zabbixfk]

[SOLVED] Zabbix-proxy and data arrival delay ( more than 2 hrs delay)

Thanks for the reply.

Well, after a couple of weeks of observation, and tweeking ( changed the HDD of server with greater I/O ops, changing item polling intervals etc), now i am able to get values properly.

Thanks.