Front end shows "Zabbix server not running"

[dbisdorf]

I've just finished setting up Zabbix under Fedora 17, but the web front end is giving me the message "Zabbix server not running: the information displayed may not be current." This seems to suggest that the front end is unable to communicate with the Zabbix server, but I can't see why that would be. MySQL and the Zabbix server are definitely running, and they're at the location the server should be looking at.

Can anyone suggest a course of investigation I can pursue?

Here are the contents of my zabbix.conf.php file:

<?php
// Zabbix GUI configuration file
global $DB;

$DB['TYPE']     = 'MYSQL';
$DB['SERVER']   = 'localhost';
$DB['PORT']     = '3306';
$DB['DATABASE'] = 'zabbix';
$DB['USER']     = 'root';
$DB['PASSWORD'] = '';

// SCHEMA is relevant only for IBM_DB2 database
$DB['SCHEMA'] = '';

$ZBX_SERVER      = '127.0.0.1';
$ZBX_SERVER_PORT = '10051';
$ZBX_SERVER_NAME = 'CIM Test';

$IMAGE_FORMAT_DEFAULT = IMAGE_FORMAT_PNG;
?>

Here's the results of "netstat -lnptu", showing that the Zabbix server and MySQL are running at the ports I've configured:

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:10051           0.0.0.0:*               LISTEN      29721/zabbix_server 
tcp        0      0 0.0.0.0:3306            0.0.0.0:*               LISTEN      9482/mysqld

I've tried using "localhost" instead of "127.0.0.1" in zabbix.conf.php to indicate the location of the Zabbix server, with the same results.

Thanks,
Don

[f.koch]

Hi,

how many trapper have you configured in your zabbix_server.conf? you need at least 1 , then the frontend won't complain.

regards

[dbisdorf]

how many trapper have you configured in your zabbix_server.conf?

I believe I have 5 trappers configured. Here's the relevant section of my zabbix_server.conf:

### Option: StartTrappers
#       Number of pre-forked instances of trappers.
#       Trappers accept incoming connections from Zabbix sender, active agents, active proxies and child nodes.
#       At least one trapper process must be running to display server availability in the frontend.
#
# Mandatory: no
# Range: 0-1000
# Default:
StartTrappers=5

[tchjts1]

Any pertinent entries in zabbix_server.log?

[dbisdorf]

Here's what my zabbix_server.log looks like after a clean start. The only thing that looks suspicious to me is the "unreachable poller" message given by server #8. Is this at all pertinent, and if so, what can I do to correct it?

 32579:20130507:092413.364 Starting Zabbix Server. Zabbix 2.0.6 (revision 35158).
 32579:20130507:092413.389 ****** Enabled features ******
 32579:20130507:092413.389 SNMP monitoring:           YES
 32579:20130507:092413.389 IPMI monitoring:            NO
 32579:20130507:092413.389 WEB monitoring:            YES
 32579:20130507:092413.389 Jabber notifications:       NO
 32579:20130507:092413.389 Ez Texting notifications:  YES
 32579:20130507:092413.389 ODBC:                       NO
 32579:20130507:092413.389 SSH2 support:              YES
 32579:20130507:092413.389 IPv6 support:              YES
 32579:20130507:092413.389 ******************************
 32583:20130507:092413.544 server #1 started [configuration syncer #1]
 32584:20130507:092413.545 server #2 started [db watchdog #1]
 32593:20130507:092414.059 server #9 started [trapper #1]
 32594:20130507:092414.064 server #10 started [trapper #2]
 32603:20130507:092414.128 server #18 started [http poller #1]
 32598:20130507:092414.117 server #13 started [trapper #5]
 32610:20130507:092414.324 server #22 started [history syncer #3]
 32600:20130507:092414.336 server #15 started [alerter #1]
 32614:20130507:092414.412 server #24 started [escalator #1]
 32615:20130507:092414.413 server #25 started [proxy poller #1]
 32613:20130507:092414.418 server #23 started [history syncer #4]
 32579:20130507:092414.418 server #0 started [main process]
 32599:20130507:092414.421 server #14 started [icmp pinger #1]
 32616:20130507:092414.421 server #26 started [self-monitoring #1]
 32602:20130507:092414.421 server #17 started [timer #1]
 32601:20130507:092414.444 server #16 started [housekeeper #1]
 32601:20130507:092414.444 executing housekeeper
 32609:20130507:092414.455 server #21 started [history syncer #2]
 32595:20130507:092414.324 server #11 started [trapper #3]
 32590:20130507:092414.576 server #6 started [poller #4]
 32608:20130507:092414.543 server #20 started [history syncer #1]
 32596:20130507:092414.594 server #12 started [trapper #4]
 32586:20130507:092414.627 server #3 started [poller #1]
 32601:20130507:092414.780 housekeeper deleted: 0 records from history and trends, 0 records of deleted items, 0 events, 0 alerts, 0 sessions
 32589:20130507:092414.843 server #5 started [poller #3]
 32588:20130507:092415.104 server #4 started [poller #2]
 32607:20130507:092415.143 server #19 started [discoverer #1]
 32591:20130507:092415.179 server #7 started [poller #5]
 32592:20130507:092415.204 server #8 started [unreachable poller #1]

[f.koch]

do you use selinux?

[tchjts1]

Here's what my zabbix_server.log looks like after a clean start. The only thing that looks suspicious to me is the "unreachable poller" message given by server #8. Is this at all pertinent, and if so, what can I do to correct it?

 32579:20130507:092413.364 Starting Zabbix Server. Zabbix 2.0.6 (revision 35158).
 32579:20130507:092413.389 ****** Enabled features ******
 32579:20130507:092413.389 SNMP monitoring:           YES
 32579:20130507:092413.389 IPMI monitoring:            NO
 32579:20130507:092413.389 WEB monitoring:            YES
 32579:20130507:092413.389 Jabber notifications:       NO
 32579:20130507:092413.389 Ez Texting notifications:  YES
 32579:20130507:092413.389 ODBC:                       NO
 32579:20130507:092413.389 SSH2 support:              YES
 32579:20130507:092413.389 IPv6 support:              YES
 32579:20130507:092413.389 ******************************
 32583:20130507:092413.544 server #1 started [configuration syncer #1]
 32584:20130507:092413.545 server #2 started [db watchdog #1]
 32593:20130507:092414.059 server #9 started [trapper #1]
 32594:20130507:092414.064 server #10 started [trapper #2]
 32603:20130507:092414.128 server #18 started [http poller #1]
 32598:20130507:092414.117 server #13 started [trapper #5]
 32610:20130507:092414.324 server #22 started [history syncer #3]
 32600:20130507:092414.336 server #15 started [alerter #1]
 32614:20130507:092414.412 server #24 started [escalator #1]
 32615:20130507:092414.413 server #25 started [proxy poller #1]
 32613:20130507:092414.418 server #23 started [history syncer #4]
 32579:20130507:092414.418 server #0 started [main process]
 32599:20130507:092414.421 server #14 started [icmp pinger #1]
 32616:20130507:092414.421 server #26 started [self-monitoring #1]
 32602:20130507:092414.421 server #17 started [timer #1]
 32601:20130507:092414.444 server #16 started [housekeeper #1]
 32601:20130507:092414.444 executing housekeeper
 32609:20130507:092414.455 server #21 started [history syncer #2]
 32595:20130507:092414.324 server #11 started [trapper #3]
 32590:20130507:092414.576 server #6 started [poller #4]
 32608:20130507:092414.543 server #20 started [history syncer #1]
 32596:20130507:092414.594 server #12 started [trapper #4]
 32586:20130507:092414.627 server #3 started [poller #1]
 32601:20130507:092414.780 housekeeper deleted: 0 records from history and trends, 0 records of deleted items, 0 events, 0 alerts, 0 sessions
 32589:20130507:092414.843 server #5 started [poller #3]
 32588:20130507:092415.104 server #4 started [poller #2]
 32607:20130507:092415.143 server #19 started [discoverer #1]
 32591:20130507:092415.179 server #7 started [poller #5]
 32592:20130507:092415.204 server #8 started [unreachable poller #1]

That all looks normal.

Any chance SElinux is enabled on your server? It has been known to cause some issues... You can run # getenforce to see the status

(Edit) - LOL. f.koch and I had the same idea...

[dbisdorf]

Yes. If it helps, my /etc/selinux/config file looks like this:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

And my /etc/selinux/semanage.conf file looks like this:

# Authors: Jason Tang <[email protected]>
#
# Copyright (C) 2004-2005 Tresys Technology, LLC
#
#  This library is free software; you can redistribute it and/or
#  modify it under the terms of the GNU Lesser General Public
#  License as published by the Free Software Foundation; either
#  version 2.1 of the License, or (at your option) any later version.
#
#  This library is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
#  Lesser General Public License for more details.
#
#  You should have received a copy of the GNU Lesser General Public
#  License along with this library; if not, write to the Free Software
#  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
#
# Specify how libsemanage will interact with a SELinux policy manager.
# The four options are:
#
#  "source"     - libsemanage manipulates a source SELinux policy
#  "direct"     - libsemanage will write directly to a module store.
#  /foo/bar     - Write by way of a policy management server, whose
#                 named socket is at /foo/bar.  The path must begin
#                 with a '/'.
#  foo.com:4242 - Establish a TCP connection to a remote policy
#                 management server at foo.com.  If there is a colon
#                 then the remainder is interpreted as a port number;
#                 otherwise default to port 4242.
module-store = direct

# When generating the final linked and expanded policy, by default
# semanage will set the policy version to POLICYDB_VERSION_MAX, as
# given in <sepol/policydb.h>.  Change this setting if a different
# version is necessary.
#policy-version = 19

# expand-check check neverallow rules when executing all semanage commands.
# Large penalty in time if you turn this on.  
expand-check=0

# usepasswd check tells semanage to scan all pass word records for home directories
# and setup the labeling correctly.  If this is turned off, SELinux will label /home 
# correctly only.  You will need to use semanage fcontext command.  
# For example, if you had home dirs in /althome directory you would have to execute
# semanage fcontext -a -e /home /althome
usepasswd=False
bzip-small=true
bzip-blocksize=5
ignoredirs=/root

[dbisdorf]

Output of "getenforce" confirms that selinux is turned on:

[root@monitor-test selinux]# getenforce
Enforcing

[tchjts1]

That could very well be your problem.

To temporarily disable it, you can run # setenforce Permissive

[Alan]

Try the followings commands :

semanage port -a -t http_port_t -p tcp 10051
setsebool httpd_can_network_connect=1

It works for me

Alan

[dbisdorf]

Yes, disabling selinux did seem to allow the front end to communicate with the Zabbix server.

In the long term, and in a production environment, I would not want to have selinux turned off. Can you suggest what changes I would need to make in order to prevent selinux from blocking this particular communication?

EDIT: Alan responded just as I was typing this post. I'll try his settings, unless any other settings would be more appropriate.

Thanks to everyone who responded!

[dbisdorf]

On my machine, running the setsebool command suggested by Alan was sufficient to get the front end to see the server:

setsebool httpd_can_network_connect=1

I could not, however, run the suggested semanage command, nor did it appear to be necessary. Port 10051 was already defined in semanage as a zabbix port:

[root@monitor-test selinux]# semanage port -l |grep 10051
zabbix_port_t                  tcp      10051

And I could not redefine it:

[root@monitor-test selinux]# semanage port -a -t http_port_t -p tcp 10051
/sbin/semanage: Port tcp/10051 already defined

But since the front end seems to work fine without defining port 10051 as an httpd port, I think this command is unnecessary. In any case, wouldn't this semanage command allow httpd to listen to 10051? Which seems wrong to me; I think the front end needs to send to 10051, while the zabbix server needs to listen to 10051, which is how I believe my system is currently configured, based on the output of semanage.

[trikke76]

don't forget to add the -P option to setsebool

else u will lose the configuration after your reboot


setsebool -P httpd_can_network_connect on


normally it should be enough to run this line and only give access to port 10051

semanage port -a -t http_port_t -p tcp 10051


if u still get the error that te server is not running check if the server is running

chkconfig zabbix-server status

[dbisdorf]

Yes, I found out about the -P option for setsebool the hard way. Everything on my test server seems to be working correctly now. As I mentioned previously, I didn't need to run "semanage port -a -t http_port_t -p tcp 10051"; I assume some other part of the setup process had defined this zabbix port already.