Zabbix agent unreachable

[jcanapa]

I have a Zabbix on a Windows 2008 R2 Hyper-V server that has been running correctly for months. Starting last night the agent becomes unreachable for five to up to thirty minutes at a time and I cannot figure out why. All of the virtual machines on the server also have agents installed and those continue to work even while the agent on the host is down.

Worst of all the agent log gives no indication that communication was lost.

Any ideas of where to start looking?

[benbugohit]

Worst of all the agent log gives no indication that communication was lost.

Any ideas of where to start looking?

Did you try setting up agents in "debug" mode so you can see logs ?
What about server logs too ?
You didn't give your zabbix version also..

[jcanapa]

It is version 2.0.4.

I have set the agent to debug mode and its logs look identical to other servers that are working correctly. For example:

5796:20130523:154007.044 In send_buffer() host:'192.168.3.153' port:10051 values:0/100
5796:20130523:154007.044 End of send_buffer():SUCCEED
5796:20130523:154007.044 Sleeping for 1 second(s)
2608:20130523:154007.044 In collect_perfstat()
2608:20130523:154008.058 In collect_perfstat()
5796:20130523:154008.058 In send_buffer() host:'192.168.3.153' port:10051 values:0/100
5796:20130523:154008.058 End of send_buffer():SUCCEED
5796:20130523:154008.058 Sleeping for 1 second(s)
2608:20130523:154009.072 In collect_perfstat()
5796:20130523:154009.072 In send_buffer() host:'192.168.3.153' port:10051 values:0/100
5796:20130523:154009.072 End of send_buffer():SUCCEED
5796:20130523:154009.072 Sleeping for 1 second(s)
5796:20130523:154010.086 In send_buffer() host:'192.168.3.153' port:10051 values:0/100
2608:20130523:154010.086 In collect_perfstat()
5796:20130523:154010.086 End of send_buffer():SUCCEED

There are no error messages in the agent log indicating a communication problem but the server sends me a warning that the agent cannot be pinged and no new information is added to the graphs.

Where are the server logs stored? I would love to take a look at them.

[tchjts1]

Where are the server logs stored? I would love to take a look at them.

Log in to your Zabbix server and look in your zabbix_server.conf file. Look for this section that will tell you where your log file is located:

### Option: LogFile
#       Name of log file.
#       If not set, syslog is used.
#
# Mandatory: no
# Default:
# LogFile=

LogFile=/var/log/zabbix/zabbix_server.log

[jcanapa]

Watching the tail is bounces back and forth between connection restored and another network error.

I can ping the IP address from the Zabbix server. Just at a loss as to why it agent can't stay connected.

[tchjts1]

How many hosts are you monitoring?

There are 2 settings you could adjust that may help with this:

In your zabbix_server.conf, increase the number of pollers. "StartPollers" Default is 5. Try increasing that to 10 or 15 and see if that helps.

The other setting that may give you some relief is to increment the "Timeout" setting. By default it is set to 3. Try bringing that to 10.

You could make both of these changes in parallel without any adverse effects.

Any changes you make to zabbix_server.conf requires you to restart the Zabbix server process.

[jcanapa]

Thanks, but still no luck.

The server it is trying to communicate with has multiple IPs. I have tried assigning them one at a time, and finally simultaneously, to the host. The result is always the same. I have also added a different host to make sure that it is not the Zabbix server itself that is the issue and the new host communicates just fine.

It has to be something with the agent or the server itself, which makes me a bit nervous...