zabbix agent don't see some process

[batosai]

Hi again,

Another problem with processes not seen. MySQL this time. I created a template for monitoring mysql and applied it on two hosts. One host says there is one proc running. The other one says there is none.

The only difference I could find in 'ps -ef' output is the time column :

Host with good result :
mysql    13901 13861  0 16:04 pts/0    00:00:00 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --port=3306 --socket=/var/run/mysqld/mysqld.sock
Host with bad result :
mysql    30704 30663  6 16:45 pts/0    00:00:09 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --port=3306 --socket=/var/run/mysqld/mysqld.sock

The host that is causing problem is the Zabbix server. Both server and clients are v1.6.5 (revision 7442).

I'm lost...

[Calimero]

what does
/path/to/zabbix_agentd -t proc.num[mysqld]
return on the faulty host ?

[batosai]

Wow, that's weird :

# zabbix_agentd -t proc.num[mysqld]
proc.num[mysqld]                              [u|1]

# telnet xx.xx.xx.xx 10050
Trying xx.xx.xx.xx...
Connected to xx.xx.xx.xx.
Escape character is '^]'.
proc.num[mysqld]
ZBXD

0Connection closed by foreign host.

How is it possible that the agent answers differently locally and from network ?

[Calimero]

Do you have grsec or other similar security patches/systems (SELinux maybe) ?

What if you "su - zabbix" and the try zabbix_agentd -t ?
What if you "ps" as zabbix ?

[batosai]

Do you have grsec or other similar security patches/systems (SELinux maybe) ?

Yes grsec.

What if you "su - zabbix" and the try zabbix_agentd -t ?

zabbix@sup:~$ /usr/sbin/zabbix_agentd -t
zabbix_agentd [31151]: zbx_option requires an argument -- t
usage: zabbix_agentd [-Vhp] [-c <file>] [-t <metric>]
zabbix@sup:~$ /usr/sbin/zabbix_agentd -t 1
1 [m|ZBX_NOTSUPPORTED]

What if you "ps" as zabbix ?

I only see zabbix-related processes.


So that was it : I rebooted on a non-grsec kernel and everything went fine. I didn't knew why I didn't like this stuff, now I've got a reason. Thanks for your help.

[Calimero]

zabbix@sup:~$ /usr/sbin/zabbix_agentd -t
zabbix_agentd [31151]: zbx_option requires an argument -- t
usage: zabbix_agentd [-Vhp] [-c <file>] [-t <metric>]
zabbix@sup:~$ /usr/sbin/zabbix_agentd -t 1
1 [m|ZBX_NOTSUPPORTED]

I meant running zabbix_agentd -t proc.num[mysql]

But anyway we have the culprit !

So that was it : I rebooted on a non-grsec kernel and everything went fine. I didn't knew why I didn't like this stuff, now I've got a reason. Thanks for your help.

I vaguely remember that there's a "privileged" group set at build time that has extended permissions.
Not sure though.

Anyway I usually end up with the solution you chose: remove grsec altogether...

Those patches require a lot of maintenance and study and if you're not willing to go that (expensive) route, I think they hurt more than they protect.