Zabbix trapper port

[quasi_modo]

Good morning everybody,

Last week I had to upgrade a Zabbix server from 2.2.1 to 2.4.3 version.
I have follow Zabbix upgrade procedure (beginning with a dump of my database) and it works fine, my server is now in 2.4.3 version.
But something disturbs me.
In fact, when I would like to run a script (ping/traceroute/detect operating system), Zabbix sends me an error :
ERROR: Cannot connect to the trapper port of zabbix server daemon, but it should be available to run the script.
And then below Unknown Host ID [10144].

Zabbix sends the same error for evey host in my network.
However, these differents host ID are present in my database "hosts".
My server is listening port 10051, and scripts work fine when I run them directly on my server.

Hoping someone can help me,


Thank you !

[jan.garaj]

I guess it's SELinux (if SELinux is presented on your system).
Or maybe Zabbix trapper processes are too busy - check Zabbix data gathering process busy graph

[quasi_modo]

Good morning Jan, and thank you for your reply.

I've check SELinux first, and this process is already disabled (nano /etc/selinux/config SELINUX=disabled).

Then I've check the Zabbix data gathering process busy graph and this one doesn't return anything since 01/22, day I made Zabbix upgrade (I mean that there is no data in this graph since this day, but there are some before).

On my network I have another Zabbix server, 2.2.1 version. It runs normally and the same graph returns some data.
But maybe having two zabbix server on the same network can create a conflict ?

[jan.garaj]

Nope, 2 Zabbix servers are not a problem (if they are configured correctly).

Check your Zabbix server/frontends configs.
Unknown Host ID [10144]. -> maybe your frontend 1 has Host ID from Zabbix DB 1, but Zabbix server is configured against Zabbix DB 2.

Check also your zabbix server config related to trappers (ListenIP, ListenPort, StartTrappers, TrapperTimeout, ...)

Zabbix proxy can be problem, if it's presented.

[quasi_modo]

So, all my datas are stored into a database named Zabbix.
This database is correctly configured on zabbix.conf.php and zabbix_server.conf files.

About trappers on my zabbix server config file :
- the line about ListenIP is commented (# ListenIP=127.0.0.1)
- the ListenPort is 10051
- the line about StartTrappers is commented (# StartTrappers=5)
- the line about TrapperTimeout is also commented (# TrapperTimeout=300)

Finally zabbix proxy is not configured here.

[filipp.sudanov]

Some details about how this functionality works, may be it will help.

When you click "ping" in the front-end, your zabbix web server will establish TCP connection to a Zabbix server trapper. Trapper is the process, that serves connections from active zabbix agents (and proxies), so you need a number of them to be running (and according to your config it should be so). Trapper listens on TCP port 10051.

When that connection is established, the following text is sent (script id and hostid might be different):

{"request":"command","scriptid":"1","hostid":"10084"}

Then trapper should reply with the output of the command, web server would render that output and return to the browser.
For testing you can try telnetting to trapper port and pasting the above command. And you should get a result:

$ telnet localhost 10051
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
{"request":"command","scriptid":"1","hostid":"10084"}
{"response":"success","data":"PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.\n64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.033 ms\n64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.044 ms\n64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.042 ms\n\n--- 127.0.0.1 ping statistics ---\n3 packets transmitted, 3 received, 0% packet loss, time 1998ms\nrtt min/avg/max/mdev = 0.033/0.039/0.044/0.008 ms\n"}Connection closed by foreign host.

[quasi_modo]

Good morning filipp !

Thank you for your help and your precisions.

The first thing I've test when I've seen my error message was telnet localhost 10051 to verify if the connection is well established.
And it runs...

So I'm sure that 10051 port is open on my server.

[filipp.sudanov]

When telnet connect, try pasting

{"request":"command","scriptid":"1","hostid":"10144"}

into it's window and paste the result here.

[quasi_modo]

Sorry I didn't understand your request before

I've paste your command after open a telnet window and it returns me :

{"response":"failed","info":"Unknown Host ID [10144]."}

[filipp.sudanov]

You've mentioned that you have two zabbix servers. Make sure that there's no confusion here.

There's a front-end configuration file (created when you do the Installation in the front-end) located where the front-end files are in conf/zabbix.conf.php

These two params specify to which zabbix trapper front-end will connect to perform ping and other operations:

$DB['SERVER']   = 'localhost';
$DB['PORT']     = '0';

[quasi_modo]

This configuration file is located in /usr/local/zabbix/frontend/conf for both zabbix servers. File's name is zabbix.conf.php.

On both zabbix.conf.php I find

$DB['SERVER']   = 'localhost';
$DB['PORT']     = '0';

[filipp.sudanov]

And if you connect to database of your Zabbix installation and run this:

select * from hosts where hostid = 10144;

what is the result?

[quasi_modo]

When I run this command I get the following informations about this host :

| hostid | proxy_hostid | host               | status | disable_until | error | available | errors_from | lastaccess | ipmi_authtype | ipmi_privilege | ipmi_username | ipmi_password | ipmi_disable_until | ipmi_available | snmp_disable_until | snmp_available | maintenanceid | maintenance_status | maintenance_type | maintenance_from | ipmi_errors_from | snmp_errors_from | ipmi_error | snmp_error | jmx_disable_until | jmx_available | jmx_errors_from | jmx_error | name               | flags | templateid | description |
|  10144 |         NULL | RI -6 150.1.48.106 |      0 |             0 |       |         0 |           0 |          0 |            -1 |              2 |               |               |                  0 |              0 |                  0 |              1 |          NULL |                  0 |                0 |                0 |                0 |                0 |            |            |                 0 |             0 |               0 |           | RI -6 150.1.48.106 |     0 |       NULL | NULL        |

[filipp.sudanov]

Strange....

Next possbile option is to set zabbix server debug level to 4 and see if there is anything related in it's log when you perform a ping.

[Parasin]

Have you checked IP tables for potential problems?