MYSQL - Database not available

[Lirria]

Hello All!

Just installed Zabbix and so far it's been great - but I find that MYSQL keeps stopping (I get an email saying MySQL database "zabbix" on "localhost" is not available: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)


If I restart MYSQL it's ok - but not sure why it keeps stopping - any advice would be greatly appreciated.

I have the server running in Azure with tunnels to all the networks that I need to have monitored - the install was done on Ubuntu 18.04, following the directions on the website.

Really appreciate any suggestions you might have.

Thank you!

Lirria

[tim.mooney]

Based on your description, it sounds like it's not really an issue with Zabbix, it's an issue with your MySQL database backend.

I would start by looking in two places:

1. The MySQL server is probably writing a log file somewhere, likely /var/log/mysql/<something> but it's configurable so it might be elsewhere. Find that log file and see if there are any useful log messages in there, especially error codes that you can use in web searches to find what the MySQL issue might be.
2. Assuming Ubuntu 18.04 uses systemd (I don't know if it does, just a guess), you can use the 'systemctl status -l' command with your mysql service unit (probably mysql.service but again, it varies so you'll have to figure that out) to get long (-l) status output on what the state of the service is. Run that after you get the message that mysql is not running. It should show you what systemd knows about why the service stopped.

PS: always helpful to include the version of Zabbix you're using, as well as the type and version of the database backend you're using, and whether it's on the same host or they're split onto separate hosts.

[Lirria]

Doh! I started to do that and just spaced out - my bad.

Zabbix 5
Mysql 5.7.31

the logs aren't really showing anything that alarms me - but I did just step back through the configuration and think I missed a setting so I'll see if that helps.

Thank you!

Lirria

[Lirria]

So no matter what I seem to update and change the Mysql keeps stopping - and now I get to see this error:

• Connection to Zabbix server "127.0.0.1" refused. Possible reasons:
• 1. Incorrect server IP/DNS in the "zabbix.conf.php";
• 2. Security environment (for example, SELinux) is blocking the connection;
• 3. Zabbix server daemon not running;
• 4. Firewall is blocking TCP connection.
• Connection refused

I've checked the ip in the zabbix.conf.php, selinux is not active, the server daemon is running, the firewall is set so now I'm stuck - I have no idea what to do

[tim.mooney]

Where are you seeing that error? In your browser when you try to view the Zabbix front-end web interface? If so, that's a problem with you web config, your web server isn't running or is being blocked by a firewall, etc. Check your web access logs and error logs in that case.

If the error message you're seeing is in your zabbix_agentd.log on one or more of your clients, that's a different type of problem.

Knowing where that message is being reported while help determine how best to debug it.

Regarding your continuing mysql issue, I'm not sure what's going on with your system, but after mysql stops the next time, run the command I suggested in step 2 in my first response, to see what systemd has to say about the mysql status.

[Lirria]

So in the front-end interface - on the queue overview page I get the following message at the top of the page in a red box
Details Cannot display item queue.

• Connection to Zabbix server "127.0.0.1" refused. Possible reasons:
• 1. Incorrect server IP/DNS in the "zabbix.conf.php";
• 2. Security environment (for example, SELinux) is blocking the connection;
• 3. Zabbix server daemon not running;
• 4. Firewall is blocking TCP connection.
• Connection refused

and at the bottom in a yellow box I get Zabbix servier is not running: the information displayed may not be current

The second message comes and goes

The problem with Mysql, once it stops, the server unfortunately becomes non-responsive - so its difficult to log in

right now the number one message I'm seeing is this:
2020-07-31T12:51:35.998572Z 491 [Note] Aborted connection 491 to db: 'zabbix' user: 'zabbix' host: 'localhost' (Got an error reading communication packets)


I'm so confused - and I'll admit that I'm not that proficient with Linux

Lirria

[tim.mooney]

Thanks for clarifying.

The web interface (Zabbix frontend) is running, but the zabbix_server is not and periodically the MySQL database is not.

If the server is unresponsive when MySQL is stopped, it may be an issue with resources on the VM. When you can log into the system, check the system logs (probably /var/log/syslog) and look for messages involving mysql. Also, look for messages involving the letters "oom". I'm wondering if the out-of-memory (oom) killer is taking out processes to try free virtual memory. If so, that's a sign that what you have running on the VM instance is overtaxing the VM you've sized.

Also, if the zabbix_serverd is not running, there may be messages in /var/log/zabbix/zabbix_serverd.log indicating why it stopped. It's probably secondary to the MySQL issue, but it may offer some clue that's useful.

[Lirria]

Thank you for pointing out what should have been the first thing to check - I didn't think about it - I just setup the VM in Azure and it well took the defaults from when I was testing things, So the server only has 1gb of ram - no wonder the poor thing is constantly crashing!


I have upped the ram and will monitor - THANK YOU!

[Lirria]

So I had to change the CacheSize and the ValueCacheSize to get the server to start up - I'll watch them and see if things are better.

Thank you for the assist I think I'm good now.

Lirria