Zabbix agent and server not communicating

[kovas]

Hi,

I have moved zabbix 3.2.5 server to cloud.
All looks good, all services are running and zabbix server works fine.
The problem that Zabbix server and Windows Agent not communicating in the SAME network.



Agent config:

LogFile=c:\Zabbix\zabbix_agentd.log
DebugLevel=5
Server=10.10.10.11
Hostname=Windows host

Agent log file:

6696:20180525:153934.855 Zabbix Agent stopped. Zabbix 3.2.7 (revision 70289).
28996:20180525:153939.693 Starting Zabbix Agent [Windows host]. Zabbix 3.2.7 (revision 70289).
28996:20180525:153939.699 **** Enabled features ****
28996:20180525:153939.703 IPv6 support: YES
28996:20180525:153939.707 TLS support: NO
28996:20180525:153939.712 **************************
28996:20180525:153939.716 using configuration file: C:\Zabbix\zabbix_agentd.conf
28996:20180525:153939.721 In init_collector_data()
28996:20180525:153939.726 End of init_collector_data()
28996:20180525:153939.730 In init_perf_collector()
28996:20180525:153939.734 End of init_perf_collector():SUCCEED
28996:20180525:153939.738 agent #0 started [main process]
25616:20180525:153939.743 agent #1 started [collector]
19956:20180525:153939.747 agent #2 started[listener #1]
12132:20180525:153939.752 agent #3 started[listener #2]
22012:20180525:153939.756 agent #4 started[listener #3]
25616:20180525:153939.760 In init_cpu_collector()
25616:20180525:153939.765 In get_counter_name() pdhIndex:238
25616:20180525:153940.148 End of get_counter_name():SUCCEED
25616:20180525:153940.160 In get_counter_name() pdhIndex:6
25616:20180525:153940.165 End of get_counter_name():SUCCEED
25616:20180525:153940.169 In add_perf_counter() counter:'\Processor(_Total)\% Processor Time' interval:900
25616:20180525:153940.175 add_perf_counter(): PerfCounter '\Processor(_Total)\% Processor Time' successfully added
25616:20180525:153940.179 End of add_perf_counter(): SUCCEED
25616:20180525:153940.184 In add_perf_counter() counter:'\Processor(0)\% Processor Time' interval:900
25616:20180525:153940.189 add_perf_counter(): PerfCounter '\Processor(0)\% Processor Time' successfully added
25616:20180525:153940.194 End of add_perf_counter(): SUCCEED
25616:20180525:153940.198 In add_perf_counter() counter:'\Processor(1)\% Processor Time' interval:900
25616:20180525:153940.203 add_perf_counter(): PerfCounter '\Processor(1)\% Processor Time' successfully added
25616:20180525:153940.207 End of add_perf_counter(): SUCCEED
25616:20180525:153940.211 In add_perf_counter() counter:'\Processor(2)\% Processor Time' interval:900
25616:20180525:153940.216 add_perf_counter(): PerfCounter '\Processor(2)\% Processor Time' successfully added
25616:20180525:153940.220 End of add_perf_counter(): SUCCEED
25616:20180525:153940.233 In add_perf_counter() counter:'\Processor(3)\% Processor Time' interval:900
25616:20180525:153940.237 add_perf_counter(): PerfCounter '\Processor(3)\% Processor Time' successfully added
25616:20180525:153940.241 End of add_perf_counter(): SUCCEED
25616:20180525:153940.246 In get_counter_name() pdhIndex:2
25616:20180525:153940.250 End of get_counter_name():SUCCEED
25616:20180525:153940.255 In get_counter_name() pdhIndex:44
25616:20180525:153940.260 End of get_counter_name():SUCCEED
25616:20180525:153940.264 In add_perf_counter() counter:'\System\Processor Queue Length' interval:900
25616:20180525:153940.275 add_perf_counter(): PerfCounter '\System\Processor Queue Length' successfully added
25616:20180525:153940.279 End of add_perf_counter(): SUCCEED
25616:20180525:153940.283 End of init_cpu_collector():SUCCEED
25616:20180525:153940.288 In collect_perfstat()
25616:20180525:153940.295 End of collect_perfstat()
25616:20180525:153941.301 In collect_perfstat()
25616:20180525:153941.312 End of collect_perfstat()
25616:20180525:153942.318 In collect_perfstat()
25616:20180525:153942.329 End of collect_perfstat()
25616:20180525:153943.333 In collect_perfstat()
25616:20180525:153943.343 End of collect_perfstat()
25616:20180525:153944.349 In collect_perfstat()
25616:20180525:153944.361 End of collect_perfstat()
25616:20180525:153945.367 In collect_perfstat()

Network is the same.
Zabbix server looks fine.
Agent looks fine.

Any way I could test command line connection between server and client?
Any other ideas?

Thanks

[HaveDill]

What happens if you do run this from the server:
zabbix_get -s HOSTNAME -k 'system.hostname'

This should return the hostname from the agent if the server can communicate with it.

In the reverse, on the client you could run:
telnet 10.10.10.11 10050

and it shouldn't error out

[kloczek]

Clarification: using zabbix_get make sense only when it is about testing zabbix passive agent setup (which is the case because Server=10.10.10.11).
However despite using passive agent setup it is quite possible that in monitored items are only "zabbix agent (active)" type items and this is why there is no monitoring data.

[kovas]

HaveDill
Yes, I am able to get answer Hostname from server. And also saw request in verbose client's log.
Also I see values by running: zabbix_get -s 100.100.100.249 -k "system.cpu.load[all,avg1]"
Also telnet from client works to server on port 10051, but server does not answer to 10050.

Problem that on Zabbix server for same client I only see: Get value from agent failed: cannot connect to [[10.10.1.29]:10050]: [111] Connection refused
And also no data in Latest Data section for that client.

kloczek
yes, at least would be good passive setup would be working in that case,

Looks like something wrong with server as data is not refreshed?

[kloczek]

Looks like something wrong with server as data is not refreshed?

I would more likely bet that it is something related to network layer traffic filtering rules.
Try test connectivity using telnet command.

[kovas]

all firewalls seems to be off, what other rules we can check?

telnet from client to server works on port 10051
telnet from server to client works on port 10050

Now I get:
Get value from agent failed: ZBX_TCP_WRITE() timed out

Any suggestions?

Many thanks

[kloczek]

So question is only what kind of the items type you have in this host zabbix settings? "zabbix agent" (passive) of "zabbix agent (active)" (active one) and what you see in the agent and server logs?
I'm almost sure that your case is only about some wrong settings if communication is possible in both directions is possible.

[kovas]

looks like if I am using DNS name in Agent interfaces in clients Host config on zabbix server he gives error, but then using IP all works. Any ideas why Host errors with FQDN?

ping from zabbix server to client server works fine by FQDN or host name

Thanks

[kloczek]

You did not answer on the question which one agent setup you are using.
Depends on this question you should be checking for FQDN resolution availability on server or agent side.

[kovas]

We use passive agents everywhere.

Ping to FQDN in agent and server works fine.

As now looks like issue is on server side I haven't checked server logs, where I could find those?

Many thanks

[kloczek]

/var/log/zabbix/zabbix_serber.log

[kovas]

it was DNS issue
thank you kloczek for help