ZABBIX Forums  

Go Back   ZABBIX Forums > Zabbix Discussions and Feedback > Zabbix Troubleshooting and Problems

Reply
 
Thread Tools Display Modes
  #1  
Old 21-11-2017, 14:43
DiViNe DiViNe is offline
Junior Member
 
Join Date: Feb 2015
Posts: 15
Lightbulb Zabbix server is not running: the iformation may not be current

Problem:
++++++++
  • Zabbix Webfrontend displays this message:
    Zabbix server is not running: the iformation may not be current
  • "Zabbix server is running" No is displayed in the webfrontend as well.
  • Webfrontend is accessible and displays new problems.
  • Zabbix-server log contains this message: 1704:20171119:031406.248 failed to accept an incoming connection: connection rejected, getpeername() failed: [107] Transport endpoint is not connected



Config:
+++++++
CentOS Linux release 7.4.1708 (Core)

zabbix-agent.x86_64 3.4.3-1.el7 @zabbix
zabbix-get.x86_64 3.4.3-1.el7 @zabbix
zabbix-java-gateway.x86_64 3.4.3-1.el7 @zabbix
zabbix-release.noarch 3.4-2.el7 @zabbix
zabbix-sender.x86_64 3.4.3-1.el7 @zabbix
zabbix-server-mysql.x86_64 3.4.3-1.el7 @zabbix
zabbix-web.noarch 3.4.3-1.el7 @zabbix
zabbix-web-mysql.noarch 3.4.3-1.el7 @zabbix

SELinux and Firewall are testwise disabled

echo $HOSTNAME
chdcvzabbix01.our-OLD-domain.name.domainending


Zabbix Processes:
+++++++++++++++++
Code:
ps -ef | grep -i zab
zabbix    1138     1  0 11:05 ?        00:00:00 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
zabbix    1144  1138  0 11:05 ?        00:00:00 /usr/sbin/zabbix_agentd: collector [idle 1 sec]
zabbix    1147  1138  0 11:05 ?        00:00:00 /usr/sbin/zabbix_agentd: listener #1 [waiting for connection]
zabbix    1149  1138  0 11:05 ?        00:00:00 /usr/sbin/zabbix_agentd: listener #2 [waiting for connection]
zabbix    1150  1138  0 11:05 ?        00:00:00 /usr/sbin/zabbix_agentd: listener #3 [waiting for connection]
zabbix    1151  1138  0 11:05 ?        00:00:00 /usr/sbin/zabbix_agentd: active checks #1 [idle 1 sec]
zabbix    1466     1  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server -c /etc/zabbix/zabbix_server.conf
zabbix    1664  1466  0 11:05 ?        00:00:01 /usr/sbin/zabbix_server: configuration syncer [synced configuration in 0.501025 sec, idle 60 sec]
zabbix    1665  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: alerter #1 started
zabbix    1666  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: housekeeper [startup idle for 30 minutes]
zabbix    1667  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: timer #1 [processed 0 triggers, 0 events in 0.000016 sec, 0 maintenances in 0.012494 sec, idle 30 sec]
zabbix    1668  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: http poller #1 [got 0 values in 0.000682 sec, idle 5 sec]
zabbix    1669  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: discoverer #1 [processed 0 rules in 0.000622 sec, idle 60 sec]
zabbix    1670  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: history syncer #1 [synced 16 items in 0.001925 sec, idle 1 sec]
zabbix    1672  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: history syncer #2 [synced 0 items in 0.000007 sec, idle 1 sec]
zabbix    1674  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: history syncer #3 [synced 0 items in 0.000054 sec, idle 1 sec]
zabbix    1676  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: history syncer #4 [synced 0 items in 0.000006 sec, idle 1 sec]
zabbix    1678  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: escalator #1 [processed 0 escalations in 0.000839 sec, idle 3 sec]
zabbix    1679  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: proxy poller #1 [exchanged data with 0 proxies in 0.000003 sec, idle 5 sec]
zabbix    1680  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: self-monitoring [processed data in 0.000006 sec, idle 1 sec]
zabbix    1681  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: vmware collector #1 [updated 0, removed 0 VMware services in 0.000001 sec, idle 5 sec]
zabbix    1682  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: vmware collector #2 [updated 0, removed 0 VMware services in 0.000002 sec, idle 5 sec]
zabbix    1684  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: task manager [processed 0 task(s) in 0.000424 sec, idle 5 sec]
zabbix    1686  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: poller #1 [got 5 values in 0.001862 sec, idle 1 sec]
zabbix    1688  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: poller #2 [got 5 values in 0.001888 sec, idle 1 sec]
zabbix    1689  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: poller #3 [got 6 values in 0.001923 sec, idle 1 sec]
zabbix    1690  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: unreachable poller #1 [got 0 values in 0.000005 sec, idle 5 sec]
zabbix    1691  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: trapper #1 [processing data]
zabbix    1692  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: trapper #2 [processing data]
zabbix    1693  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: trapper #3 [processing data]
zabbix    1695  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: icmp pinger #1 [got 0 values in 0.000048 sec, idle 1 sec]
zabbix    1697  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: icmp pinger #2 [got 0 values in 0.000042 sec, idle 1 sec]
zabbix    1698  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: alert manager #1 [sent 0, failed 0 alerts, idle 5.009370 sec during 5.009372 sec]
zabbix    1699  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: preprocessing manager #1 [queued 0, processed 15 values, idle 5.004293 sec during 5.004424 sec]
zabbix    1700  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: preprocessing worker #1 started
zabbix    1701  1466  0 11:05 ?        00:00:00 /usr/sbin/zabbix_server: preprocessing worker #2 started
root      2086  1770  0 11:16 pts/0    00:00:00 grep --color=auto -i zab
Open Ports from User Zabbix
+++++++++++++++++++++++++++
Code:
 lsof -a -i -u zabbix
COMMAND    PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
zabbix_ag 1138 zabbix    4u  IPv4  16236      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_ag 1138 zabbix    5u  IPv6  16237      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_ag 1144 zabbix    4u  IPv4  16236      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_ag 1144 zabbix    5u  IPv6  16237      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_ag 1147 zabbix    4u  IPv4  16236      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_ag 1147 zabbix    5u  IPv6  16237      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_ag 1149 zabbix    4u  IPv4  16236      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_ag 1149 zabbix    5u  IPv6  16237      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_ag 1150 zabbix    4u  IPv4  16236      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_ag 1150 zabbix    5u  IPv6  16237      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_ag 1151 zabbix    4u  IPv4  16236      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_ag 1151 zabbix    5u  IPv6  16237      0t0  TCP *:zabbix-agent (LISTEN)
zabbix_se 1466 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1466 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1664 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1664 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1665 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1665 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1666 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1666 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1667 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1667 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1668 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1668 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1669 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1669 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1670 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1670 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1672 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1672 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1674 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1674 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1676 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1676 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1678 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1678 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1679 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1679 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1680 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1680 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1681 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1681 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1682 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1682 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1684 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1684 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1686 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1686 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1688 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1688 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1689 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1689 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1690 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1690 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1691 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1691 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1691 zabbix    8u  IPv4  25696      0t0  TCP our-domani:zabbix-trapper->10.189.15.21:49422 (CLOSE_WAIT)
zabbix_se 1691 zabbix    9u  IPv4  23202      0t0  UDP our-domani:37547->servername2.our-domani:domain
zabbix_se 1692 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1692 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1692 zabbix    8u  IPv4  23193      0t0  TCP our-domani:zabbix-trapper->10.189.15.22:60518 (CLOSE_WAIT)
zabbix_se 1692 zabbix    9u  IPv4  23201      0t0  UDP our-domani:53823->servername2.our-domani:domain
zabbix_se 1693 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1693 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1693 zabbix    8u  IPv4  23197      0t0  TCP our-domani:zabbix-trapper->10.171.75.11:50300 (CLOSE_WAIT)
zabbix_se 1693 zabbix    9u  IPv4  23198      0t0  UDP our-domani:37602->servername1.our-domani:domain
zabbix_se 1693 zabbix   10u  IPv4  23200      0t0  UDP our-domani:45733->servername2.our-domani:domain
zabbix_se 1695 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1695 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1697 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1697 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1698 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1698 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1699 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1699 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1700 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1700 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1701 zabbix    5u  IPv4  18198      0t0  TCP *:zabbix-trapper (LISTEN)
zabbix_se 1701 zabbix    6u  IPv6  18199      0t0  TCP *:zabbix-trapper (LISTEN)

Zabbix-Agent (systemctl)
++++++++++++++++++++++++
Code:
systemctl status zabbix-agent
● zabbix-agent.service - Zabbix Agent
   Loaded: loaded (/usr/lib/systemd/system/zabbix-agent.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2017-11-21 11:26:39 CET; 13s ago
  Process: 2221 ExecStop=/bin/kill -SIGTERM $MAINPID (code=exited, status=0/SUCCESS)
  Process: 2394 ExecStart=/usr/sbin/zabbix_agentd -c $CONFFILE (code=exited, status=0/SUCCESS)
 Main PID: 2396 (zabbix_agentd)
   CGroup: /system.slice/zabbix-agent.service
           ├─2396 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
           ├─2397 /usr/sbin/zabbix_agentd: collector [idle 1 sec]
           ├─2398 /usr/sbin/zabbix_agentd: listener #1 [waiting for connection]
           ├─2399 /usr/sbin/zabbix_agentd: listener #2 [waiting for connection]
           ├─2400 /usr/sbin/zabbix_agentd: listener #3 [waiting for connection]
           └─2401 /usr/sbin/zabbix_agentd: active checks #1 [idle 1 sec]

Nov 21 11:26:39 chdcvzabbix01our-domani systemd[1]: Starting Zabbix Agent...
Nov 21 11:26:39 chdcvzabbix01our-domani systemd[1]: Started Zabbix Agent.
After a while:

Code:
systemctl status zabbix-agent
● zabbix-agent.service - Zabbix Agent
   Loaded: loaded (/usr/lib/systemd/system/zabbix-agent.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2017-11-21 12:56:55 CET; 28min ago
  Process: 4120 ExecStop=/bin/kill -SIGTERM $MAINPID (code=exited, status=0/SUCCESS)
  Process: 4123 ExecStart=/usr/sbin/zabbix_agentd -c $CONFFILE (code=exited, status=0/SUCCESS)
 Main PID: 4125 (zabbix_agentd)
   CGroup: /system.slice/zabbix-agent.service
           ├─4125 /usr/sbin/zabbix_agentd -c /etc/zabbix/zabbix_agentd.conf
           ├─4126 /usr/sbin/zabbix_agentd: collector [idle 1 sec]
           ├─4127 /usr/sbin/zabbix_agentd: listener #1 [waiting for connection]
           ├─4128 /usr/sbin/zabbix_agentd: listener #2 [waiting for connection]
           ├─4129 /usr/sbin/zabbix_agentd: listener #3 [waiting for connection]
           └─4130 /usr/sbin/zabbix_agentd: active checks #1 [idle 1 sec]

Nov 21 12:56:55 chdcvzabbix01.ch.viamat.com systemd[1]: Starting Zabbix Agent...
Nov 21 12:56:55 chdcvzabbix01.ch.viamat.com systemd[1]: zabbix-agent.service: Supervising process 4125 which is not our child. We'll most likely not notice when it exits.
Nov 21 12:56:55 chdcvzabbix01.ch.viamat.com systemd[1]: Started Zabbix Agent.


Zabbix-Agent (journalctl)
++++++++++++++++++++++++
Code:
journalctl -u zabbix-agent
-- Logs begin at Tue 2017-11-21 11:28:30 CET, end at Tue 2017-11-21 11:29:16 CET. --
Nov 21 11:28:40 chdcvzabbix01.our-domani systemd[1]: Starting Zabbix Agent...
Nov 21 11:28:40 chdcvzabbix01.our-domani systemd[1]: PID file /run/zabbix/zabbix_agentd.pid not readable (yet?) after start.
Nov 21 11:28:40 chdcvzabbix01.our-domani systemd[1]: Started Zabbix Agent.

----> ls -l /run/zabbix/zabbix_agentd.pid
-rw-rw-r-- 1 zabbix zabbix 4 Nov 21 11:28 /run/zabbix/zabbix_agentd.pid


Zabbix-Server (systemctl)
+++++++++++++++++++++++++
Code:
 systemctl status zabbix-server
● zabbix-server.service - Zabbix Server
   Loaded: loaded (/usr/lib/systemd/system/zabbix-server.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2017-11-21 11:28:41 CET; 2min 5s ago
  Process: 1070 ExecStart=/usr/sbin/zabbix_server -c $CONFFILE (code=exited, status=0/SUCCESS)
 Main PID: 1512 (zabbix_server)
   CGroup: /system.slice/zabbix-server.service
           ├─1512 /usr/sbin/zabbix_server -c /etc/zabbix/zabbix_server.conf
           ├─1668 /usr/sbin/zabbix_server: configuration syncer [synced configuration in 0.523490 sec, idle 60 sec]
           ├─1669 /usr/sbin/zabbix_server: alerter #1 started
           ├─1670 /usr/sbin/zabbix_server: housekeeper [startup idle for 30 minutes]
           ├─1671 /usr/sbin/zabbix_server: timer #1 [processed 0 triggers, 0 events in 0.000082 sec, 0 maintenances in 0.000000 sec, idle 30 sec]
           ├─1673 /usr/sbin/zabbix_server: http poller #1 [got 0 values in 0.000674 sec, idle 5 sec]
           ├─1675 /usr/sbin/zabbix_server: discoverer #1 [processed 0 rules in 0.000513 sec, idle 60 sec]
           ├─1676 /usr/sbin/zabbix_server: history syncer #1 [synced 3 items in 0.003086 sec, idle 1 sec]
           ├─1677 /usr/sbin/zabbix_server: history syncer #2 [synced 0 items in 0.000001 sec, idle 1 sec]
           ├─1679 /usr/sbin/zabbix_server: history syncer #3 [synced 0 items in 0.000002 sec, idle 1 sec]
           ├─1682 /usr/sbin/zabbix_server: history syncer #4 [synced 0 items in 0.000001 sec, idle 1 sec]
           ├─1684 /usr/sbin/zabbix_server: escalator #1 [processed 0 escalations in 0.000687 sec, idle 3 sec]
           ├─1685 /usr/sbin/zabbix_server: proxy poller #1 [exchanged data with 0 proxies in 0.000003 sec, idle 5 sec]
           ├─1686 /usr/sbin/zabbix_server: self-monitoring [processed data in 0.000006 sec, idle 1 sec]
           ├─1687 /usr/sbin/zabbix_server: vmware collector #1 [updated 0, removed 0 VMware services in 0.000002 sec, idle 5 sec]
           ├─1688 /usr/sbin/zabbix_server: vmware collector #2 [updated 0, removed 0 VMware services in 0.000004 sec, idle 5 sec]
           ├─1689 /usr/sbin/zabbix_server: task manager [processed 0 task(s) in 0.000475 sec, idle 5 sec]
           ├─1691 /usr/sbin/zabbix_server: poller #1 [got 1 values in 0.000689 sec, idle 1 sec]
           ├─1692 /usr/sbin/zabbix_server: poller #2 [got 1 values in 0.000678 sec, idle 1 sec]
           ├─1693 /usr/sbin/zabbix_server: poller #3 [got 1 values in 0.000561 sec, idle 1 sec]
           ├─1695 /usr/sbin/zabbix_server: unreachable poller #1 [got 0 values in 0.000004 sec, idle 5 sec]
           ├─1696 /usr/sbin/zabbix_server: trapper #1 [processing data]
           ├─1697 /usr/sbin/zabbix_server: trapper #2 [processing data]
           ├─1698 /usr/sbin/zabbix_server: trapper #3 [processing data]
           ├─1699 /usr/sbin/zabbix_server: icmp pinger #1 [got 0 values in 0.000112 sec, idle 1 sec]
           ├─1700 /usr/sbin/zabbix_server: icmp pinger #2 [got 0 values in 0.000113 sec, idle 1 sec]
           ├─1701 /usr/sbin/zabbix_server: alert manager #1 [sent 0, failed 0 alerts, idle 5.009563 sec during 5.009565 sec]
           ├─1702 /usr/sbin/zabbix_server: preprocessing manager #1 [queued 0, processed 22 values, idle 5.004756 sec during 5.004926 sec]
           ├─1703 /usr/sbin/zabbix_server: preprocessing worker #1 started
           └─1704 /usr/sbin/zabbix_server: preprocessing worker #2 started

Nov 21 11:28:40 chdcvzabbix01.our-domani systemd[1]: Starting Zabbix Server...
Nov 21 11:28:41 chdcvzabbix01.our-domani systemd[1]: Started Zabbix Server.

Zabbix-Server (journalctl)
+++++++++++++++++++++++++
Code:
 journalctl -u zabbix-server
-- Logs begin at Tue 2017-11-21 11:28:30 CET, end at Tue 2017-11-21 11:29:16 CET. --
Nov 21 11:28:40 chdcvzabbix01.our-domani systemd[1]: Starting Zabbix Server...
Nov 21 11:28:41 chdcvzabbix01.our-domani systemd[1]: Started Zabbix Server.




Zabbix Server Conf
+++++++++++++++++++
Code:
grep ^[^#] /etc/zabbix/zabbix_server.conf
LogFile=/var/log/zabbix/zabbix_server.log
LogFileSize=128
DebugLevel=3
PidFile=/var/run/zabbix/zabbix_server.pid
SocketDir=/var/run/zabbix
DBHost=localhost
DBName=zabbixdb
DBUser=zabbixuser
DBPassword=OurPassword
StartPollers=3
StartIPMIPollers=0
StartPreprocessors=2
StartPollersUnreachable=1
StartTrappers=3
StartPingers=2
StartDiscoverers=1
StartHTTPPollers=1
StartTimers=1
StartEscalators=1
StartAlerters=1
StartJavaPollers=0
StartVMwareCollectors=2
VMwareFrequency=600
VMwarePerfFrequency=300
VMwareCacheSize=48M
VMwareTimeout=10
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
CacheSize=128M
CacheUpdateFrequency=60
StartDBSyncers=4
HistoryCacheSize=48M
HistoryIndexCacheSize=24M
TrendCacheSize=24M
ValueCacheSize=24M
Timeout=4
AlertScriptsPath=/usr/lib/zabbix/alertscripts
ExternalScripts=/usr/lib/zabbix/externalscripts
LogSlowQueries=3000

Zabbix Agent Conf
+++++++++++++++++++
Code:
PidFile=/var/run/zabbix/zabbix_agentd.pid
LogFile=/var/log/zabbix/zabbix_agentd.log
LogFileSize=128
DebugLevel=4
Server=10.170.12.240 (This was 127.0.0.1 before,but this way the agent did not work. So I changed it with the ip)
ServerActive=10.170.12.240 (This was 127.0.0.1 before,but this way the agent did not work. So I changed it with the ip)
HostnameItem=system.hostname
Include=/etc/zabbix/zabbix_agentd.d/*.conf

Zabbix-Server Log: /var/log/zabbix/zabbix_server
+++++++++++++++++++++++++++++++++++++++++++++++++
Code:
1704:20171119:031349.818 failed to accept an incoming connection: connection rejected, getpeername() failed: [107] Transport endpoint is not connected


Zabbix-Server Log: /var/log/zabbix/zabbix_agentd.log
+++++++++++++++++++++++++++++++++++++++++++++++++
Code:
8684:20171121:105617.859 Starting Zabbix Agent [chdcvzabbix01.our-olddomain]. Zabbix 3.4.3 (revision 73588).
  8684:20171121:105617.859 **** Enabled features ****
  8684:20171121:105617.859 IPv6 support:          YES
  8684:20171121:105617.859 TLS support:           YES
  8684:20171121:105617.859 **************************
  8684:20171121:105617.859 using configuration file: /etc/zabbix/zabbix_agentd.conf
  8684:20171121:105617.859 In zbx_load_modules()
  8684:20171121:105617.859 End of zbx_load_modules():SUCCEED
  8684:20171121:105617.859 In init_collector_data()
  8684:20171121:105617.859 In zbx_dshm_create() size:0
  8684:20171121:105617.859 End of zbx_dshm_create():SUCCEED shmid:-1
  8684:20171121:105617.859 End of init_collector_data()
  8684:20171121:105617.859 agent #0 started [main process]
  8686:20171121:105617.862 agent #2 started[listener #1]
  8686:20171121:105617.862 In zbx_tls_init_child()
  8686:20171121:105617.863 OpenSSL library (version OpenSSL 1.0.1e-fips 11 Feb 2013) initialized
  8686:20171121:105617.863 End of zbx_tls_init_child()
  8686:20171121:105617.863 __zbx_zbx_setproctitle() title:'listener #1 [waiting for connection]'
  8688:20171121:105617.863 agent #4 started[listener #3]
  8688:20171121:105617.864 In zbx_tls_init_child()
  8688:20171121:105617.865 OpenSSL library (version OpenSSL 1.0.1e-fips 11 Feb 2013) initialized
  8688:20171121:105617.865 End of zbx_tls_init_child()
  8688:20171121:105617.865 __zbx_zbx_setproctitle() title:'listener #3 [waiting for connection]'
  8689:20171121:105617.867 agent #5 started [active checks #1]
  8689:20171121:105617.867 In zbx_tls_init_child()
  8689:20171121:105617.869 OpenSSL library (version OpenSSL 1.0.1e-fips 11 Feb 2013) initialized
  8689:20171121:105617.869 End of zbx_tls_init_child()
  8689:20171121:105617.869 In init_active_metrics()
  8689:20171121:105617.869 buffer: first allocation for 100 elements
  8689:20171121:105617.869 End of init_active_metrics()
  8689:20171121:105617.869 In send_buffer() host:'127.0.0.1' port:10051 entries:0/100
  8689:20171121:105617.869 End of send_buffer():SUCCEED
  8689:20171121:105617.869 __zbx_zbx_setproctitle() title:'active checks #1 [getting list of active checks]'
  8689:20171121:105617.869 In refresh_active_checks() host:'127.0.0.1' port:10051
  8687:20171121:105617.869 agent #3 started[listener #2]
  8687:20171121:105617.869 In zbx_tls_init_child()
  8687:20171121:105617.871 OpenSSL library (version OpenSSL 1.0.1e-fips 11 Feb 2013) initialized
  8685:20171121:105617.871 agent #1 started [collector]
  8687:20171121:105617.871 End of zbx_tls_init_child()
  8689:20171121:105617.871 active check configuration update from [127.0.0.1:10051] started to fail (cannot connect to [[127.0.0.1]:10051]: [111] Connection refused)
  8685:20171121:105617.872 In init_cpu_collector()
  8689:20171121:105617.872 End of refresh_active_checks():FAIL
  8685:20171121:105617.872 End of init_cpu_collector():SUCCEED
  8687:20171121:105617.872 __zbx_zbx_setproctitle() title:'listener #2 [waiting for connection]'
  8689:20171121:105617.872 __zbx_zbx_setproctitle() title:'active checks #1 [processing active checks]'
  8689:20171121:105617.872 In process_active_checks() server:'127.0.0.1' port:10051
  8685:20171121:105617.872 __zbx_zbx_setproctitle() title:'collector [processing data]'
  8689:20171121:105617.872 End of process_active_checks()
  8685:20171121:105617.872 In update_cpustats()
  8689:20171121:105617.872 In get_min_nextcheck()
  8685:20171121:105617.872 End of update_cpustats()
  8689:20171121:105617.873 End of get_min_nextcheck():-1
  8685:20171121:105617.873 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
  8689:20171121:105617.873 __zbx_zbx_setproctitle() title:'active checks #1 [idle 1 sec]'
  8689:20171121:105618.873 In send_buffer() host:'127.0.0.1' port:10051 entries:0/100
  8689:20171121:105618.873 End of send_buffer():SUCCEED
  8685:20171121:105618.873 __zbx_zbx_setproctitle() title:'collector [processing data]'
  8689:20171121:105618.873 __zbx_zbx_setproctitle() title:'active checks #1 [idle 1 sec]'
  8685:20171121:105618.873 In update_cpustats()
  8685:20171121:105618.873 End of update_cpustats()
  8685:20171121:105618.873 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
  8689:20171121:105619.873 In send_buffer() host:'127.0.0.1' port:10051 entries:0/100
  8689:20171121:105619.873 End of send_buffer():SUCCEED
  8689:20171121:105619.873 __zbx_zbx_setproctitle() title:'active checks #1 [idle 1 sec]'
  8685:20171121:105619.873 __zbx_zbx_setproctitle() title:'collector [processing data]'
  8685:20171121:105619.873 In update_cpustats()
  8685:20171121:105619.874 End of update_cpustats()
  8685:20171121:105619.874 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
  8689:20171121:105620.874 In send_buffer() host:'127.0.0.1' port:10051 entries:0/100
  8689:20171121:105620.874 End of send_buffer():SUCCEED
  8689:20171121:105620.874 __zbx_zbx_setproctitle() title:'active checks #1 [idle 1 sec]'
  8685:20171121:105620.874 __zbx_zbx_setproctitle() title:'collector [processing data]'
  8685:20171121:105620.874 In update_cpustats()
  8685:20171121:105620.874 End of update_cpustats()
  8685:20171121:105620.874 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
  8689:20171121:105621.874 In send_buffer() host:'127.0.0.1' port:10051 entries:0/100
  8689:20171121:105621.874 End of send_buffer():SUCCEED
  8689:20171121:105621.874 __zbx_zbx_setproctitle() title:'active checks #1 [idle 1 sec]'
  8685:20171121:105621.874 __zbx_zbx_setproctitle() title:'collector [processing data]'
  8685:20171121:105621.874 In update_cpustats()
  8685:20171121:105621.874 End of update_cpustats()
  8685:20171121:105621.874 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
  8689:20171121:105622.874 In send_buffer() host:'127.0.0.1' port:10051 entries:0/100
  8689:20171121:105622.874 End of send_buffer():SUCCEED
  8689:20171121:105622.874 __zbx_zbx_setproctitle() title:'active checks #1 [idle 1 sec]'
  8685:20171121:105622.875 __zbx_zbx_setproctitle() title:'collector [processing data]'
  8685:20171121:105622.875 In update_cpustats()
  8685:20171121:105622.875 End of update_cpustats()
  8685:20171121:105622.875 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
  8689:20171121:105623.874 In send_buffer() host:'127.0.0.1' port:10051 entries:0/100
  8689:20171121:105623.874 End of send_buffer():SUCCEED
  8689:20171121:105623.875 __zbx_zbx_setproctitle() title:'active checks #1 [idle 1 sec]'
  8685:20171121:105623.875 __zbx_zbx_setproctitle() title:'collector [processing data]'
  8685:20171121:105623.875 In update_cpustats()
  8685:20171121:105623.875 End of update_cpustats()
  8685:20171121:105623.875 __zbx_zbx_setproctitle() title:'collector [idle 1 sec]'
  8689:20171121:105624.875 In send_buffer() host:'127.0.0.1' port:10051 entries:0/100
  8689:20171121:105624.875 End of send_buffer():SUCCEED
  8689:20171121:105624.875 __zbx_zbx_setproctitle() title:'active checks #1 [idle 1 sec]'
  8685:20171121:105624.876 __zbx_zbx_setproctitle() title:'collector [processing data]'
  8685:20171121:105624.876 In update_cpustats()

/etc/zabbix/web/zabbix.conf.php
+++++++++++++++++++++++++++++++
Code:
  <?php
// Zabbix GUI configuration file.
global $DB;

$DB['TYPE']     = 'MYSQL';
$DB['SERVER']   = 'localhost';
$DB['PORT']     = '0';
$DB['DATABASE'] = 'zabbixdb';
$DB['USER']     = 'zabbixuser';
$DB['PASSWORD'] = 'OurPW';

// Schema name. Used for IBM DB2 and PostgreSQL.
$DB['SCHEMA'] = '';

$ZBX_SERVER      = 'localhost';
$ZBX_SERVER_PORT = '10051';
$ZBX_SERVER_NAME = 'Zabbix Monitoring';

$IMAGE_FORMAT_DEFAULT = IMAGE_FORMAT_PNG;



/etc/httpd/conf/httpd.conf (not all)
++++++++++++++++++++++++++++++++++++
Code:
EnableSendfile on
IncludeOptional conf.d/*.conf
<VirtualHost *:443>
        SSLEngine on
        SSLCertificateFile /etc/pki/tls/certs/domain.crt
        SSLCertificateKeyFile /etc/pki/tls/private/domain.key
        <Directory /usr/share/zabbix/>
        AllowOverride All
        </Directory>
        DocumentRoot /usr/share/zabbix/
        ServerName our-other-domain.name.com
</VirtualHost>
NameVirtualHost *:80
<VirtualHost *:80>
DocumentRoot /usr/share/zabbix/
ServerName out-other-domain
    <IfModule mod_rewrite.c>
      RewriteEngine On
      RewriteCond %{HTTPS} off
      RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
    </IfModule>
</VirtualHost>
Reply With Quote
  #2  
Old 22-11-2017, 11:56
dimir dimir is offline
Zabbix developer
 
Join Date: Apr 2011
Location: Riga, Latvia
Posts: 48
Default

You mention in Zabbix Agent Conf:

Code:
Server=10.170.12.240 (This was 127.0.0.1 before,but this way the agent did not work. So I changed it with the ip)
So perhaps try setting the same IP for frontend, /etc/zabbix/web/zabbix.conf.php:

Code:
$ZBX_SERVER      = '10.170.12.240';
Reply With Quote
  #3  
Old 23-11-2017, 11:33
DiViNe DiViNe is offline
Junior Member
 
Join Date: Feb 2015
Posts: 15
Red face

Dear dimir

Thanks a lot for you replay!

I had "localhost" in both configs and changed it recently to 10.170.12.240' (but not in both configs -thanks for the info). I changed it now in both, but it still doesn’t work.

Regards

DiViNe
Reply With Quote
  #4  
Old 23-11-2017, 11:37
dimir dimir is offline
Zabbix developer
 
Join Date: Apr 2011
Location: Riga, Latvia
Posts: 48
Default

What do you get if you perform from the command-line:

Code:
telnet 10.170.12.240 10051
or this one:

Code:
telnet 127.0.0.1 10051
also this one please:

Code:
/sbin/ifconfig
Reply With Quote
  #5  
Old 28-11-2017, 13:20
DiViNe DiViNe is offline
Junior Member
 
Join Date: Feb 2015
Posts: 15
Default

here we go:

Code:
[root@chdcvzabbix01 ~]#telnet 10.170.12.240 10051
Trying 10.170.12.240...
Connected to 10.170.12.240.
Escape character is '^]'.


[root@chdcvzabbix01 ~]#telnet 127.0.0.1 10051
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.


[root@chdcvzabbix01 ~]#/sbin/ifconfig
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.170.12.240  netmask 255.255.255.0  broadcast 10.170.12.255
        inet6 fe80::250:56ff:fe87:a8fd  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:87:a8:fd  txqueuelen 1000  (Ethernet)
        RX packets 90611516  bytes 7293763950 (6.7 GiB)
        RX errors 0  dropped 2044  overruns 0  frame 0
        TX packets 96339415  bytes 10642285259 (9.9 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1  (Local Loopback)
        RX packets 4171639  bytes 248051024 (236.5 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4171639  bytes 248051024 (236.5 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I tried it also with NC:

Code:
[root@chdcvzabbix01 ~]#nc -v 10.170.12.240 10051
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: Connected to 10.170.12.240:10051.

[root@chdcvzabbix01 ~]#nc -v 10.170.12.240 10051
Ncat: Version 6.40 ( http://nmap.org/ncat )
Ncat: Connected to 10.170.12.240:10051.
Reply With Quote
  #6  
Old 28-11-2017, 13:58
dimir dimir is offline
Zabbix developer
 
Join Date: Apr 2011
Location: Riga, Latvia
Posts: 48
Default

Do you have SELinux/grsecurity enabled? Could they be causing an issue?
Reply With Quote
  #7  
Old 28-11-2017, 14:06
DiViNe DiViNe is offline
Junior Member
 
Join Date: Feb 2015
Posts: 15
Default

No:

Code:
grep ^[^#] /etc/sysconfig/selinux
SELINUX=disabled
SELINUXTYPE=targeted
I've also disabled testwise the firewall....
Reply With Quote
  #8  
Old 28-11-2017, 14:39
dimir dimir is offline
Zabbix developer
 
Join Date: Apr 2011
Location: Riga, Latvia
Posts: 48
Default

Maybe your Zabbix server trapper process is so busy all the time it cannot handle requests from frontend. In that case you could try increasing the number of trapper processes (you have StartTrappers=3).

Also make sure the frontend you are using uses the configuration file you specified. As it looks like frontend is has Zabbix server/port misconfigured.
Reply With Quote
  #9  
Old 29-11-2017, 13:35
DiViNe DiViNe is offline
Junior Member
 
Join Date: Feb 2015
Posts: 15
Arrow Changed Trapper and looking for frontend-config

Thank you for your input!

Quote:
Originally Posted by dimir View Post
Maybe your Zabbix server trapper process is so busy all the time it cannot handle requests from frontend. In that case you could try increasing the number of trapper processes (you have StartTrappers=3).
I increased it up to 500 (and restarted the server) but it didn't work.

Quote:
Also make sure the frontend you are using uses the configuration file you specified. As it looks like frontend is has Zabbix server/port misconfigured.
Do you mean this file?
/etc/zabbix/web/zabbix.conf.php
And what should I change?

Thanks you
Reply With Quote
  #10  
Old 06-12-2017, 12:16
DiViNe DiViNe is offline
Junior Member
 
Join Date: Feb 2015
Posts: 15
Arrow

I've changed it testwise to 10050 but it didn't work, so I switchted it back to 10051

Code:
[root@chdcvzabbix01 ~]#vi /etc/zabbix/web/zabbix.conf.php
<?php
// Zabbix GUI configuration file.
global $DB;

$DB['TYPE']     = 'MYSQL';
$DB['SERVER']   = 'localhost';
$DB['PORT']     = '0';
$DB['DATABASE'] = 'zabbixdb';
$DB['USER']     = 'zabbixuser';
$DB['PASSWORD'] = 'MyCensoredPassword';

// Schema name. Used for IBM DB2 and PostgreSQL.
$DB['SCHEMA'] = '';

$ZBX_SERVER      = '10.170.12.240';
$ZBX_SERVER_PORT = '10050';
$ZBX_SERVER_NAME = 'Zabbix Monitoring';

$IMAGE_FORMAT_DEFAULT = IMAGE_FORMAT_PNG;
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 10:05.