Zabbix Proxy Error: bind() failed: [99]

[GraveSkull]

Hello my friends!

I need you help because I found no answer to my problem.

A zabbix proxy that I have installed on a client suddenly started to display the following error in the log:

# tail -f /var/log/zabbix/zabbix_proxy.log
5091:20190726:110926.327 Still unable to connect...
5092:20190726:110926.554 Unable to connect to the server [XXX.XXX.XXX.XXX]:10051 [bind() failed: [99] Cannot assign requested address]

The odd thing is that the ip in question responds to ping, but doesn't send any alarms to the zabbix server.

The main proxy settings are correct without any changes.

The SNMP, Agent, and Proxy services are functioning normally.
The version we use in Zabbix is version 3.2.11 if this information helps. Thank you for your help

[Markku]

Hi, for me "bind() failed: [99] Cannot assign requested address" looks like your zabbix_proxy.conf has an incorrect configuration in ListenIP/ListenPort/SourceIP parameter. What is the IP of proxy and how are those parameters configured?

What do you mean by "Proxy services are functioning normally", if your proxy does not send data to Zabbix server?

Markku

[GraveSkull]

Hello, Markku

Sorry for my delay.


Zabbix Proxy, Server, and Agent file settings are correct.

The problem happened from hour to hour.

And another problem I'm having is that the triggers reporting the last proxy access are alarming, but ping responds.



Follows trigger setting: {tel1zproxy1p:zabbix[proxy,{HOST.HOST},lastaccess].fuzzytime(50)}=0 I believe the problems I am having with the alarms are also due to the problem reported by the log in the first post.

[Markku]

Can you post the ”ss -s” output here when the problem is occurring?

Markku

[GraveSkull]

Follow the Output:

user@tel2zproxy2p:~$ ss -s
Total: 382 (kernel 0)
TCP: 54 (estab 1, closed 37, orphaned 0, synrecv 0, timewait 36/0), ports 0

Transport Total IP IPv6
* 0 - -
RAW 1 0 1
UDP 21 13 8
TCP 17 9 8
INET 39 22 17
FRAG 0 0 0

[Markku]

So we can say that at least the proxy’s TCP stack is not very heavily utilized. How about the server IP address, is it static, does it have multiple IP addresses, etc? Can you show the proxy config file (without passwords and all the commented lines)?

Markku

[GraveSkull]

Follows filled fields from zabbix_proxy.conf file:
# This is a configuration file for Zabbix proxy daemon
# To get more information about Zabbix, visit http://www.zabbix.com

ProxyMode=0

Server=172.19.111.100

ServerPort=10051

Hostname=tel2zproxy2p

ListenPort=10051

SourceIP=172.19.111.100

LogType=file

LogFile=/var/log/zabbix/zabbix_proxy.log

LogFileSize=0

DebugLevel=3

PidFile=/var/run/zabbix/zabbix_proxy.pid

DBName=XXXXXX

DBUser= XXXXXX

DBPassword= XXXXXX

ProxyOfflineBuffer=240

HeartbeatFrequency=10

ConfigFrequency=600

StartPollers=10

StartTrappers=5

StartPingers=10

StartDiscoverers=5

JavaGateway=172.25.132.190

JavaGatewayPort=10052

StartJavaPollers=5

StartVMwareCollectors=5

VMwareFrequency=60

VMwarePerfFrequency=60

VMwareCacheSize=8M

VMwareTimeout=10

SNMPTrapperFile=/var/log/snmptt/snmptt.log

StartSNMPTrapper=1

### Option: ListenIP
# List of comma delimited IP addresses that the trapper should listen on.
# Trapper will listen on all network interfaces if this parameter is missing.
#
# Mandatory: no
# Default:
# ListenIP=0.0.0.0


HousekeepingFrequency=1

StartDBSyncers=5

Timeout=4

TrapperTimeout=300

FpingLocation=/usr/bin/fping

Fping6Location=/usr/bin/fping6



About the low usage of TCP, I believe it is because the problem has lasted a little over a month.

[GraveSkull]

IP 172.19.111.100 is my client's NAT style IP.

I forgot to mention that.

[Markku]

Follows filled fields from zabbix_proxy.conf file:

Server=172.19.111.100

SourceIP=172.19.111.100

So, your proxy’s IP is .100 and your server’s IP is .100, that’s your problem.

Markku

[GraveSkull]

I didn't understand very well, Markku.

Could you explain further, please.

[Markku]

https://www.zabbix.com/documentation/4.0/manual/appendix/config/zabbix_proxy

Server = IP address or DNS name of Zabbix server to get configuration data from and send data to.

SourceIP = Source IP address for outgoing connections.

Your config says: ”My IP address is .100 (SourceIP) and I will try to connect to the server at address .100 (Server)”. Doesn’t make any sense because you are not running proxy on your server.

SourceIP is usually not needed. Remove it unless you have a reason to keep it, in which case please correct it. And correct the Server parameter if that is not correct, in the screenshot it is apparently 192.168.2.238.

Markku

[GraveSkull]

I found the problem

We have two proxies and both were pointing to port 10051, when I commented on the ListenIP and SourceIP lines of conf, the proxies were back.

Thanks so much for the help, Markku