Cannot get connection to the database working, so zabbix server doesn't come up

[wolpi35]

All,

I am new with Zabbix and would like to install it on a formerly Desktop system which I built now with CentOS 8.

root@oskar : /opt > cat /etc/os-release
NAME="CentOS Stream"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Stream 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
root@oskar : /opt >

It is an older system (Acer Desktop with 4 GB RAM)

As database system I choosed mariadb, where I can login as zabbix user and where I run the sql-scripts which are described on the installation description at the zabbix page (https://www.zabbix.com/de/download?z...ysql&ws=apache)

root@oskar : /opt > mysql zabbix -u zabbix -p
Enter password:
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 8
Server version: 10.7.3-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [zabbix]> show tables;
+----------------------------+
| Tables_in_zabbix |
+----------------------------+
| acknowledges |
| actions |
| alerts |
| auditlog |
| autoreg_host |
| conditions |
| config |
| config_autoreg_tls |
| corr_condition |
| corr_condition_group |
| corr_condition_tag |
| corr_condition_tagpair |
| corr_condition_tagvalue |
| corr_operation |
| correlation |
| dashboard |
| dashboard_page |
| dashboard_user |
snip ...

... snap
| role |
| role_rule |
| script_param |
| scripts |
| service_alarms |
| service_problem |
| service_problem_tag |
| service_status_rule |
| service_tag |
| services |
| services_links |
| sessions |
| sla |
| sla_excluded_downtime |
| sla_schedule |
| sla_service_tag |
| sysmap_element_trigger |
| sysmap_element_url |
| sysmap_shape |
| sysmap_url |
| sysmap_user |
| sysmap_usrgrp |
| sysmaps |
| sysmaps_element_tag |
| sysmaps_elements |
| sysmaps_link_triggers |
| sysmaps_links |
| tag_filter |
| task |
| task_acknowledge |
| task_check_now |
| task_close_problem |
| task_data |
| task_remote_command |
| task_remote_command_result |
| task_result |
| timeperiods |
| token |
| trends |
| trends_uint |
| trigger_depends |
| trigger_discovery |
| trigger_queue |
| trigger_tag |
| triggers |
| users |
| users_groups |
| usrgrp |
| valuemap |
| valuemap_mapping |
| widget |
| widget_field |
+----------------------------+
173 rows in set (0.003 sec)

MariaDB [zabbix]> quit
Bye
root@oskar : /opt >

The zabbix packages I installed are those:

root@oskar : /opt > rpm -qa | grep -i zabbix
zabbix-web-mysql-6.0.3-1.el8.noarch
zabbix-selinux-policy-6.0.3-1.el8.x86_64
zabbix-server-mysql-6.0.3-1.el8.x86_64
zabbix-release-6.0-1.el8.noarch
zabbix-agent-6.0.3-1.el8.x86_64
zabbix-web-deps-6.0.3-1.el8.noarch
zabbix-apache-conf-6.0.3-1.el8.noarch
zabbix-sql-scripts-6.0.3-1.el8.noarch
zabbix-web-6.0.3-1.el8.noarch
root@oskar : /opt >

And Apache is up and running as well:

root@oskar : /opt > systemctl status httpd.service
● httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/httpd.service.d
└─php-fpm.conf
Active: active (running) since Fri 2022-04-22 09:04:47 EDT; 1 day 21h ago
Docs: man:httpd.service(8)
Main PID: 48232 (httpd)
Status: "Running, listening on: 192.168.0.135 port 80"
Tasks: 213 (limit: 23388)
Memory: 25.3M
CGroup: /system.slice/httpd.service
├─48232 /usr/sbin/httpd -DFOREGROUND
├─99858 /usr/sbin/httpd -DFOREGROUND
├─99859 /usr/sbin/httpd -DFOREGROUND
├─99860 /usr/sbin/httpd -DFOREGROUND
└─99862 /usr/sbin/httpd -DFOREGROUND

Apr 22 09:04:47 oskar systemd[1]: httpd.service: Succeeded.
Apr 22 09:04:47 oskar systemd[1]: Stopped The Apache HTTP Server.
Apr 22 09:04:47 oskar systemd[1]: Starting The Apache HTTP Server...
Apr 22 09:04:47 oskar httpd[48232]: AH00558: httpd: Could not reliably determine the server's fully>
Apr 22 09:04:47 oskar systemd[1]: Started The Apache HTTP Server.
Apr 22 09:04:47 oskar httpd[48232]: Server configured, listening on: 192.168.0.135 port 80
Apr 24 03:12:02 oskar systemd[1]: Reloading The Apache HTTP Server.
Apr 24 03:12:02 oskar httpd[99850]: AH00558: httpd: Could not reliably determine the server's fully>
Apr 24 03:12:02 oskar systemd[1]: Reloaded The Apache HTTP Server.
Apr 24 03:12:02 oskar httpd[48232]: Server configured, listening on: 192.168.0.135 port 80
root@oskar : /opt >

But still something I don't understand is wrong, because in the logfile (/var/log/zabbix/zabbix_server.log) it is written, that zabbix cannot connect with the database?

119536:20220424:065800.200 Starting Zabbix Server. Zabbix 6.0.3 (revision 506e2b51e2).
119536:20220424:065800.200 ****** Enabled features ******
119536:20220424:065800.200 SNMP monitoring: YES
119536:20220424:065800.200 IPMI monitoring: YES
119536:20220424:065800.200 Web monitoring: YES
119536:20220424:065800.200 VMware monitoring: YES
119536:20220424:065800.200 SMTP authentication: YES
119536:20220424:065800.200 ODBC: YES
119536:20220424:065800.200 SSH support: YES
119536:20220424:065800.200 IPv6 support: YES
119536:20220424:065800.200 TLS support: YES
119536:20220424:065800.200 ******************************
119536:20220424:065800.200 using configuration file: /etc/zabbix/zabbix_server.conf
119536:20220424:065800.207 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local server through socket '/var/lib/mysql/mysql.sock' (13)
119536:20220424:065800.207 database is down: reconnecting in 10 seconds
119536:20220424:065810.208 [Z3001] connection to database 'zabbix' failed: [2002] Can't connect to local server through socket '/var/lib/mysql/mysql.sock' (13)
119536:20220424:065810.208 database is down: reconnecting in 10 seconds

My config file I cannot attach at the moment but if some information will be needed I am happy to share it!

I read already some information here and at other places I found in the web and read for example that I should put complete pathes to the mysql.sock file, and other ideas. I tried some, but still no success, so I kindly ask for instructions from more experianced guys.

Thanks and kind regards

Stefan

[Atsushi]

When Zabbix server cannot connect to the database, the error number is 13, so it seems that you do not have the permission to connect to the socket for the database. Have you enabled SELinux? Consider setting or disabling the policy for SELinux. If you are using Zabbix 6.0, installing the package zabbix-selinux-policy may help.

[wolpi35]

Hi,

Thanks for this tip! I did not realize, that SELinux is running because I work more with Debian systems ;-). To solve it I disabled SELinux via Config:

root@oskar : /etc/zabbix > sestatus
SELinux status: disabled
root@oskar : /etc/zabbix >

After this I added too the default Webserver port and the Zabbix default port, which I configured, at the firewall and so I can reach now Apache from my desktop computer as well:

root@oskar : ~ > firewall-cmd --list-all
public (active)
target: default
icmp-block-inversion: no
interfaces: enp6s0
sources:
services: cockpit dhcpv6-client ssh
ports: 10051/tcp 80/tcp
protocols:
forward: no
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
root@oskar : ~ >

What irritates me is that even the zabbix server is up and running, but the port is not listening:

root@oskar : /etc/zabbix > netstat -a | grep -i list
tcp 0 0 oskar:http 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:ssh 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:zabbix-agent 0.0.0.0:* LISTEN
tcp6 0 0 [::]:ssh [::]:* LISTEN
tcp6 0 0 [::]:zabbix-agent [::]:* LISTEN
root@oskar : /etc/zabbix >

root@oskar : ~ > systemctl status zabbix-server.service
● zabbix-server.service - Zabbix Server
Loaded: loaded (/usr/lib/systemd/system/zabbix-server.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2022-04-25 04:19:16 EDT; 2min 34s ago
Process: 754 ExecStart=/usr/sbin/zabbix_server -c $CONFFILE (code=exited, status=0/SUCCESS)
Main PID: 790 (zabbix_server)
Tasks: 1 (limit: 23399)
Memory: 8.2M
CGroup: /system.slice/zabbix-server.service
└─790 /usr/sbin/zabbix_server -c /etc/zabbix/zabbix_server.conf

Apr 25 04:19:13 oskar systemd[1]: Starting Zabbix Server...
Apr 25 04:19:16 oskar systemd[1]: Started Zabbix Server.
root@oskar : ~ >

Can you give me another tip please?

Thanks and regards

Stefan

PS: I did not install the package zabbix-selinux-policy, because I thought I reach it in a bit different way or is it still needed?

[Atsushi]

Remember to reboot the OS after setting SELinux to disabled.
One more thing I missed is that the version of MariaDB available in Zabbix 6.0 is 10.5.00-10.6.X. Please use a different DBMS or downgrade to the corresponding version.

https://www.zabbix.com/documentation/current/en/manual/installation/requirements#database-management-system

[wolpi35]

Hi Atsushi,

thanks again for your answer! The server I rebooted of course after I disabled SELLinux and by the way the command sestatus will not show "disabled" if I would not have done this. Wirth your other tip to downgrade MariaDB I did that as well - downgrade is not possible so I uninstalled the 10.7 and installed a 10.6. after. As you can see below, the version should suits now as well:

MariaDB [zabbix]> show variables like 'version';
+---------------+----------------+
| Variable_name | Value |
+---------------+----------------+
| version | 10.6.7-MariaDB |
+---------------+----------------+
1 row in set (0.002 sec)

MariaDB [zabbix]>

By the way, I checked the tables, which I imported again, so all is as before, like permissions, etc. I folllowed again the installation instructions on the part where the database was related.

The zabbix-server is up and running again, find below the status:

root@oskar : /etc/zabbix > systemctl status zabbix-server.service
● zabbix-server.service - Zabbix Server
Loaded: loaded (/usr/lib/systemd/system/zabbix-server.service; disabled; vendor preset: disabled)
Active: active (running) since Tue 2022-04-26 13:20:41 EDT; 11s ago
Process: 10948 ExecStart=/usr/sbin/zabbix_server -c $CONFFILE (code=exited, status=0/SUCCESS)
Main PID: 10950 (zabbix_server)
Tasks: 48 (limit: 23399)
Memory: 41.8M
CGroup: /system.slice/zabbix-server.service
snap ...

... snap
└─11025 /usr/sbin/zabbix_server: odbc poller Home [got 0 values in 0.000092 sec, idle 5 se>

Apr 26 13:20:41 oskar systemd[1]: Starting Zabbix Server...
Apr 26 13:20:41 oskar systemd[1]: zabbix-server.service: Can't open PID file /run/zabbix/zabbix_ser>
Apr 26 13:20:41 oskar systemd[1]: Started Zabbix Server.
root@oskar : /etc/zabbix >

But still I cannot reach the server via webinterface but at the logfile I find now some messages like this:

10995:20220426:132133.245 temporarily disabling Zabbix agent checks on host "Zabbix server": interface unavailable
10981:20220426:132141.548 item "Zabbix server:zabbix[process,ipmi poller,avg,busy]" became not supported: No "ipmi poller" processes started.
10979:20220426:132142.556 item "Zabbix server:zabbix[process,java poller,avg,busy]" became not supported: No "java poller" processes started.

I would be very happy about another tip how I can go ahead

Thanks in advance

Stefan

[Atsushi]

systemctl status may display old past information, so please check Zabbix server log to see if Zabbix server is running normally. However, it seems that the status is running, so I think that it can be started.
The poller process for IPMI and JMX is not started by default, so if you want to monitor IPMI or JMX, please set the number of each Poller in zabbix_server.conf.

ex.
StartIPMIPollers
StartJavaPollers

https://www.zabbix.com/documentation/current/en/manual/appendix/config/zabbix_server

If you don't want to monitor using IPMI or JMX, you can safely ignore the "ipmi poller" and "java poller" messages.