Perfomance problem: High CPU Usage

[proctozont]

I'm installed zabbix 1.3.8 on FreeBSD 5.5
Server has 4 CPU and 3.5 GB of RAM + 4 GB of SWAP

As resullt 6 zabbix_server process uses all avalable CPU resources.

I'm add only 1 host with about 224 items.

On another FreeBSD 6.2 server with 1 CPU and 200 MB of RAM all ok .

Any ideas?

PS:
Sorry for English

[Eugene]

I hope, this bug solved.
Try to use 1.4

[Andrew]

I hope, this bug solved.
Try to use 1.4

I see the same problem on 1.4, but with old configs (from 1.3.4)

After config rewriting (from 1.4 distr), problem gone out...

[Alexei]

Hmm, interesting. Any hint on reproducing this?

[proctozont]

I'm monitoring swiches in LAN with SNMPv3 and md5 auth + des encryption
"BIG" Server has 4 cpu, each at half slower than an "SMALL" (~4x1000Mhz).
I think that all because of DES encryption. By default snmp using as i know triple DES in CBC mode, which as I know not very fast (AES Encryption not implemented in zabbix).
Sory for that post, now I'm going to think that this is not a zabbix bug and server is realy too slow for that.
Within the next few days I'll install v 1.4 and I'll disable encryption on some test swiches and test perfomance.
It is interesting that top and ps shows that process sleping.
BIG Server:
last pid: 2021; load averages: 0.86, 0.86, 0.79 up 0+01:30:54 02:10:53
128 processes: 2 running, 126 sleeping

Mem: 101M Active, 37M Inact, 71M Wired, 652K Cache, 52M Buf, 3397M Free
Swap: 4096M Total, 4096M Free


PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND
1309 zabbix 8 5 4744K 3852K nanslp 1 5:30 13.18% 13.18% zabbix_server
1308 zabbix 8 5 4744K 3852K nanslp 1 5:20 12.55% 12.55% zabbix_server
1307 zabbix 8 5 4744K 3852K nanslp 2 5:20 12.06% 12.06% zabbix_server
1306 zabbix 8 5 4744K 3852K nanslp 1 5:20 11.33% 11.33% zabbix_server
1311 zabbix 8 5 4744K 3852K nanslp 3 5:21 10.84% 10.84% zabbix_server
1310 zabbix 102 5 4764K 3856K CPU3 1 5:29 10.25% 10.25% zabbix_server
583 mysql 20 0 67088K 40436K kserel 0 5:58 0.63% 0.63% mysqld
245 root 96 0 3736K 2516K select 3 0:32 0.54% 0.54% ppp
498 root 96 0 3364K 2772K select 3 0:18 0.05% 0.05% ssh
684 root 86 -10 60164K 16296K select 1 1:01 0.00% 0.00% utm5_core
571 root 96 0 1472K 692K select 2 0:37 0.00% 0.00% ndsad
591 www 20 0 18772K 15304K lockf 3 0:11 0.00% 0.00% httpd
587 www 20 0 18236K 14576K lockf 1 0:10 0.00% 0.00% httpd
952 root 96 0 3444K 2412K select 0 0:09 0.00% 0.00% ppp
921 root 8 0 6292K 4636K nanslp 3 0:08 0.00% 0.00% verlihub
1018 www 20 0 19812K 16320K lockf 3 0:08 0.00% 0.00% httpd
590 www 20 0 18772K 15116K lockf 1 0:07 0.00% 0.00% httpd
SMALL Server
last pid: 35842; load averages: 0.90, 0.60, 0.55 up 1+10:35:57 02:14:46
84 processes: 2 running, 82 sleeping

Mem: 112M Active, 12M Inact, 52M Wired, 11M Cache, 29M Buf, 576K Free
Swap: 303M Total, 43M Used, 259M Free, 14% Inuse


PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
845 zabbix 1 8 5 4512K 1184K nanslp 90:13 5.91% zabbix_server
843 zabbix 1 8 5 4512K 1392K nanslp 90:04 5.91% zabbix_server
844 zabbix 1 8 5 4512K 1172K nanslp 90:56 5.57% zabbix_server
846 zabbix 1 103 5 4528K 1408K RUN 89:58 5.42% zabbix_server
847 zabbix 1 8 5 4512K 1392K nanslp 89:58 4.49% zabbix_server
848 zabbix 1 8 5 4512K 1396K nanslp 89:57 4.35% zabbix_server
35836 root 1 20 0 4500K 2368K pause 0:01 3.83% csh
35833 root 1 96 0 6224K 2652K select 0:01 0.50% sshd
750 mysql 27 20 0 61660K 19036K kserel 91:05 0.24% mysqld
745 www 1 4 0 26220K 14696K accept 11:59 0.00% httpd
746 www 1 4 0 23876K 12304K accept 11:40 0.00% httpd
747 www 1 4 0 21620K 10168K accept 11:36 0.00% httpd
744 www 1 4 0 24040K 12296K accept 11:20 0.00% httpd
832 www 1 4 0 26204K 13952K accept 11:14 0.00% httpd
748 www 1 4 0 25456K 12272K accept 10:46 0.00% httpd
833 www 1 4 0 26060K 11928K accept 10:40 0.00% httpd
31804 www 1 4 0 25748K 12284K accept 10:35 0.00% httpd
32948 www 1 4 0 21756K 10844K accept 1:22 0.00% httpd

PS
As always Sorry for English

[sauron]

Zabbix use net-snmp or ucd-snmp for communicate with snmp device and AES not emplement iin net-snmp

[proctozont]

Zabbix use net-snmp or ucd-snmp for communicate with snmp device and AES not emplement iin net-snmp

If Zabbix use net-snmp library, it doesn't mean that Zabbix support all features that this library implement.

Source 1.3.8:

else if(item->snmpv3_securitylevel == ITEM_SNMPV3_SECURITYLEVEL_AUTHPRIV)
{
session.securityLevel = SNMP_SEC_LEVEL_AUTHPRIV;

/* set the authentication method to MD5 */
session.securityAuthProto = usmHMACMD5AuthProtocol;
session.securityAuthProtoLen = USM_AUTH_PROTO_MD5_LEN;
session.securityAuthKeyLen = USM_AUTH_KU_LEN;

if (generate_Ku(session.securityAuthProto,
session.securityAuthProtoLen,
(u_char *) item->snmpv3_authpassphrase, strlen(item->snmpv3_authpassphrase),
session.securityAuthKey,
&session.securityAuthKeyLen) != SNMPERR_SUCCESS)
{
zbx_snprintf(error,sizeof(error),"Error generating Ku from authentication pass phrase.");

zabbix_log( LOG_LEVEL_ERR, "%s", error);
SET_MSG_RESULT(value, strdup(error));

return NOTSUPPORTED;
}

/* set the private method to DES */
session.securityPrivProto = usmDESPrivProtocol;
session.securityPrivProtoLen = USM_PRIV_PROTO_DES_LEN;
session.securityPrivKeyLen = USM_PRIV_KU_LEN;

if (generate_Ku(session.securityAuthProto,
session.securityAuthProtoLen,
(u_char *) item->snmpv3_privpassphrase, strlen(item->snmpv3_privpassphrase),
session.securityPrivKey,
&session.securityPrivKeyLen) != SNMPERR_SUCCESS)
{
zbx_snprintf(error,sizeof(error),"Error generating Ku from priv pass phrase.");

zabbix_log( LOG_LEVEL_ERR, "%s", error);
SET_MSG_RESULT(value, strdup(error));

return NOTSUPPORTED;
}
}
zabbix_log( LOG_LEVEL_DEBUG, "SNMPv3 [%s@%s:%d]",
session.securityName,
session.peername,
session.remote_port);
}

Only MD5 and only DES.

[sauron]

net-snmp don't support AES but SHA support can be added.

[proctozont]

net-snmp don't support AES but SHA support can be added.

net-snmp support AES
And I sed that currently this features not supported.
Of course you can add everything, but now......

[sauron]

Huh is already support But for this need remove ucd-snmp support