What's the correct script command to restart systemd services?

[v1ktor]

Hi,

I'm trying to set up actions to restart some systemd services when they are down, and the command seems to be executed (confirmed in Action Log and agent log) on the server but services aren't restarting.

On the agent side, I enabled command logging and debugging at level 5. Here's what the log says:

2/04/05 05:12:54.969098 received passive check request: 'system.run[/etc/init.d/mariadb restart,nowait]' from 'xx.xx.xx.xx'
2022/04/05 05:12:54.969160 [1] processing update request (1 requests)
2022/04/05 05:12:54.969179 [1] adding new request for key: 'system.run[/etc/init.d/mariadb restart,nowait]'
2022/04/05 05:12:54.969192 [1] created direct exporter task for plugin 'SystemRun' itemid:0 key 'system.run[/etc/init.d/mariadb restart,nowait]'
2022/04/05 05:12:54.969243 executing direct exporter task for key 'system.run[/etc/init.d/mariadb restart,nowait]'
2022/04/05 05:12:54.969249 [SystemRun] Executing command:'/etc/init.d/mariadb restart'
2022/04/05 05:12:54.969483 executed direct exporter task for key 'system.run[/etc/init.d/mariadb restart,nowait]'
2022/04/05 05:12:54.969544 sending passive check response: '1' to 'xx.xx.xx.xx'

The script command I'm using to restart MariaDB (in this case) is:

/etc/init.d/mariadb restart

Is the script command correct to restart services? Or is there something else?

I'm using:

• Zabbix 6.0.2
• Zabbix_agent2
• Debian 11
• MariaDB 10.5.13

Would appreciate any help or feedback on this. Thanks.

[tim.mooney]

This is a systemd question, not a Zabbix question. You'll probably get better results doing web searching, rather than using Zabbix-specific forums.

Using /etc/init.d/<something> is the old System V init script method of starting/stopping/restarting/etc. services. On a system that's using systemd, issuing that command will (currently) be translated into syntax for systemd and passed to the systemctl command.

You want to spend some time reading the man page and web documentation for 'systemctl'. Not all systemd services are going to support a "restart" command, and even if they do, it may not be called "restart", so some investigation into each service may be necessary.

[v1ktor]

Thanks for replying Tim. I should have been more specific. /etc/init.d/ was the last thing I try, which is something I've seen in another forum post about restarting services in Zabbix. I initially tried systemctl and service, they didn't work. I also did try systemctl start. That didn't work either. Everything works fine and restarts when a root user executes it, but when Zabbix agent2 does it nothing happens. Logs show it was executed, but the services did not restart or start. So my question isn't systemd commands 101, but rather how do I format the command for Zabbix agent to execute it successfully in the scripts section. I'm missing something, but I can't figure out what it is. Google and documentation did not help.

This is what I have right now as my script:


Thanks.

[tim.mooney]

Remember that the Zabbix agent does not run as root (nor should it), it runs as a non-privileged service user (generally, "zabbix"). Because it's not root, it doesn't have permission to restart most services.

You could address that in several different ways, but the most common way to do that would be to add a file to /etc/sudoers.d/ that grants the 'zabbix' user the ability to run "/usr/bin/systemctl start" and potentially other commands without specifying a password (read the suoders(4) man page and look for the NOPASS) option. Once you have granted the zabbix user limited rights to start and restart services, then you would modify the command within Zabbix to be something like

/usr/bin/sudo /usr/bin/systemctl start mysql

In addition, you probably need to modify the zabbix_agentd.conf file (or the config file for agent2, since that's what you're using) on each client system to grant it the right to run /usr/bin/sudo. The agent config settings to grant access to specific commands have recently changed, so be sure you read the comments in the zabbix_agentd.conf for version 6 of the agent. Older comments you might find on the forums would probably specify a method that is now deprecated.

[v1ktor]

Thanks Tim, it looks like I got it to work but there may be a problem.

After I replied yesterday and mentioned that commands work if I execute them as root user, I remembered reading something about Zabbix agent and root permissions. So I followed instructions for agent2 and Zabbix 6 here, but agent2 was still not executing commands successfully. So I removed root settings, and switched to your recommendation to use sudo and that worked. Could there be a bug with root permissions in agent2? It's a new system, based on what the article says.

For reference, here's what I did to make Zabbix agent2 execute commands on the server (this is the source article, which coincidentally uses init.d commands):

1. Create a new file inside /etc/sudoers.d/ as follows:

visudo -f /etc/sudoers.d/zabbix

2. Add the following to the empty file:

# allows 'zabbix' user to run all commands without password.
zabbix ALL=NOPASSWD: ALL

This allows zabbix user to run all commands, but it can be limited to certain commands. Refer to the source article for examples.

3. That's it for the agent side. And the command in the script can be any of the examples we mentioned. I settled on:

/usr/bin/sudo /usr/bin/systemctl restart mysql

But, this will also work:

sudo systemctl restart mysql

This also works with custom systemd units/services, in case anyone is doing that.

[tim.mooney]

Not sure about agent2. My site has been using Zabbix for a long time and we haven't yet found a reason for us to switch to agent2, so we use agentd exclusively. My take on "could this be a bug" is that unexpected behavior could always be a bug, but until I've explored it pretty extensively, I always assume it's something that's misconfigured or perhaps not clear from the documentation, rather than a bug.

I'm glad you got it working, and I also appreciate you following up and showing your config. This may help others down the road.

One thing though: I'm a very strong believer in the Principle of Least Privilege, so although it's convenient to give the zabbix user the ability to run 'ALL' commands as root, that's not something I would do. I would only grant root access to the specific command(s) (and if possible, "sub commands"), not everything.

[v1ktor]

One thing though: I'm a very strong believer in the Principle of Least Privilege, so although it's convenient to give the zabbix user the ability to run 'ALL' commands as root, that's not something I would do. I would only grant root access to the specific command(s) (and if possible, "sub commands"), not everything.

Thanks. That's my plan. 'ALL' was just for testing, to make sure it actually works before I limit execution to specific commands.

[tim.mooney]

Understood. You provided such a good step-by-step guide that others will almost certainly copy it for their installations, so having a more secure example of setup for sudo was the thing I wanted to call out.

[helloguys2024]

hi guys,
I know this thread is a bit old but I am hoping you guys can help as this is the most closest thread I've seen with the issues that I am having.
zabbix server: 6.0.29 (RHEL 8.9)
zabbix_agent2: 6.0.26 (RHEL 9.3)

I am new to zabbix and trying to setup salt-minion.service to restart if the service is either failed/inactive. My script looks the same as V1ktor's as I essentially copied his config. I substituted mysql for salt-minion.


Then I created Trigger actions under Configuration-> Actions -> trigger actions


then the operations


on the client server, I modified the following files;
1. modified /etc/sudoers and added:
zabbix ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart salt-minion.service
2. modified /etc/zabbix/zabbix_agent2.conf and added:
AllowKey=system.run[*]

I restarted the zabbix-agent service.
I then stopped the salt-minion.service to test the script but I when I look on the monitor dashboard, nothing happens, I don't see any changes and when looking at zabbix server's /var/log/zabbix_server.log, this is the only entry I see
6426:20240520:172142.721 Zabbix agent item "systemd.unit.get["salt-minion.service"]" on host "test-server" failed: another network error, wait for 15 seconds

on the client server, I don't see any entry related to salt-minion server in /var/log/zabbix_agent2.log

I feel like I'm very close but not sure where to go from here.

Thanks for looking and any advice would be appreciated.