Zabbix agent show confusing messages

[miramira007]

Hellau out there,

i have some issue with my zabbix agent on linux! During i can monitor services on windows without any problems with service_state[servicename]. On Linux i have some troubleshoot. I created some Userparameter:

UserParameter=service.check[*], systemctl status $1 | grep -c running

When I'm testing "systemctl status $1 | grep -c running" in the terminal on my linux maschine i'm get 1 (service is running, 0 for all other options)

But in my front-end, i'm getting 0!

Short summary: The expression on the terminal is giving me a right return, but on the front-end i'm getting 0 and this is not the true! (zabbix agent is lying to me )

Do someone have an idea?

[Firm]

Check zabbix_agentd.log. Probably, zabbix can't run systemctl.

[miramira007]

on another host (on centos) it works fine. even with systemctl. both services are running on a centos machine. it must be another issue. another idea?

[Atsushi]

What is the CentOS version of the environment which you cannot get real value?
For example, systemd is not available on CentOS 6 by default.
systemd is available on CentOS 7.

Please tell me the CentOS version of do not work environment.

[miramira007]

In our environment we have centos 7 or redhat 7,6 (64bit version). but im confisud about the effect, that if i run the line "systectl status service | grep -c running" in the terminal. it gives me the right value. from the agent, trought the zabbix agent, it returns me a wrong value.

i tried to check the user rights: zabbix has the rights to execute all.

but if the agent cant systemctl, why it returns me a value? Shouldn't i get some error message??

cu,mira

[Firm]

What is the output of

su - zabbix -c 'systemctl status <process_name> | grep -c running'

?

[miramira007]

Hellau,

im getting this error message

su: warning: cannot change directory to /var/lib/zabbix: No such file or directory
This account is currently not available.

What does it mean? On this host my UserParameter is running and the agent gives a right value. On the other hosts i haven't got any rights. This is my test server with centos7 and here does everything fine

cu, mira

[Atsushi]

I think that can not be used systemctl command on RHEL 6.

[miramira007]

Yes i know it, systemctl is not supported in centos6. im talking about the centos7. if someone have another idee to monitor linux services, no such as "systemctl status service | grep -c running"- i liked this command much, because, it just gives me a 0 or a 1(running). I want to get such a result. But you can tell me, how you handle this.

cu, mira

[michaelwemoto]

I'm seeing the same issue with CentOS7.2, Disabling selinux allows this to work. I'm having great difficulty building a custom selinux policy that allows zabbix to use systemctl for all unit files;

zab-client;
cat /etc/zabbix/zabbix_agentd.d/custom.conf
UserParameter=custom.service.check[*],if systemctl status $1 | grep running >> /dev/null; then echo 0; else echo 1; fi

zab-server;
zabbix_get -s 192.168.59.186 -k "custom.service.check[httpd]"
sh: /usr/bin/systemctl: Permission denied
1

zab-client;
setenforce 0

zab-server;
[root@zabbix-server ~]# zabbix_get -s 192.168.59.186 -k "custom.service.check[httpd]"
1
[root@zabbix-server ~]# zabbix_get -s 192.168.59.186 -k "custom.service.check[crond]"
0
[root@zabbix-server ~]# zabbix_get -s 192.168.59.186 -k "custom.service.check[postfix]"
0
[root@zabbix-server ~]# zabbix_get -s 192.168.59.186 -k "custom.service.check[firewalld]"
0

zab-client;
audit2allow -a
#============= init_t ==============
allow init_t zabbix_agent_t:dbus send_msg;

#============= zabbix_agent_t ==============
allow zabbix_agent_t cgroup_t:file { read open };
allow zabbix_agent_t crond_unit_file_t:service status;
allow zabbix_agent_t firewalld_unit_file_t:service status;
allow zabbix_agent_t init_t:dbus send_msg;
allow zabbix_agent_t init_t:service status;
allow zabbix_agent_t system_dbusd_t:dbus send_msg;
allow zabbix_agent_t system_dbusd_t:unix_stream_socket connectto;
allow zabbix_agent_t system_dbusd_var_run_t:sock_file write;
allow zabbix_agent_t systemd_systemctl_exec_t:file execute_no_trans;
allow zabbix_agent_t systemd_unit_file_t:service status;


audit2allow -a -M zab
semodule -i zab.pp
setenforce 1

and now the services I tried to check while selnux was not enforcing are fine. I need to allow ALL unit_file_t:service status, this is where I am stuck.

any suggestions?

[michaelwemoto]

Problem solved;

client -
# reset your log so you only catch the warnings that we want
echo "" > /var/log/audit/audit.log

setenforce 0

server -
zabbix_get -s 123.456.789.1 -k "custom.service.check[firewalld]"
0

client-
cat /var/log/audit/audit.log | audit2allow -M zabbix-firewalld
semodule -i zabbix-firewalld.pp

setenforce 1

That policy will allow firewalld service status alone, I made a policy for each service I required and will only apply them where needed.

After running the above command and inspecting the zabbix-firewalld.te there are a lot of entries for just systemctl and not specifically firewalld.

I repeated the process above for another service and then split my policies;

cat zabbix-systemctl.te
module zabbix-systemctl 1.0;
require {
type cgroup_t;
type system_dbusd_var_run_t;
type init_t;
type systemd_systemctl_exec_t;
type zabbix_agent_t;
type system_dbusd_t;
class sock_file write;
class unix_stream_socket connectto;
class dbus send_msg;
class file { read execute open execute_no_trans };
class service status;
}

#============= init_t ==============
allow init_t zabbix_agent_t:dbus send_msg;

#============= zabbix_agent_t ==============
allow zabbix_agent_t cgroup_t:file { read open };
allow zabbix_agent_t init_t:dbus send_msg;
allow zabbix_agent_t system_dbusd_t:dbus send_msg;
allow zabbix_agent_t system_dbusd_t:unix_stream_socket connectto;
allow zabbix_agent_t system_dbusd_var_run_t:sock_file write;
allow zabbix_agent_t systemd_systemctl_exec_t:file { execute execute_no_trans };

cat zabbix-firewalld.te
module zabbix-firewalld 1.0;

require {
type zabbix_agent_t;
type firewalld_unit_file_t;
class service status;
}

#============= zabbix_agent_t ==============
allow zabbix_agent_t firewalld_unit_file_t:service status;

cat zabbix-postfix.te
module zabbix-postfix 1.0;

require {
type systemd_unit_file_t;
type zabbix_agent_t;
class service status;
}

#============= zabbix_agent_t ==============
allow zabbix_agent_t systemd_unit_file_t:service status;


cat zabbix-httpd.te
module zabbix-httpd 1.0;

require {
type httpd_unit_file_t;
type zabbix_agent_t;
class service status;
}


#============= zabbix_agent_t ==============
allow zabbix_agent_t httpd_unit_file_t:service status;

after creating the *.te files you follow this process to compile into a selinux policy;

checkmodule -M -m -o zabbix-systemctl.mod zabbix-systemctl.te
semodule_package -o zabbix-systemctl.pp -m zabbix-systemctl.mod
semodule -i zabbix-systemctl.pp

replacing the file names as appropriate.

I hope this helps someone out, I spent a week trying to solve this.