JMX connection failing with Unsupported or unrecognized SSL message

[j.kisters]

Hello,

i am having trouble to enable the connection between out zabbix 5.0 server and the zabbix java gateway.

The "JMX" status icon displays this message:

Unsupported or unrecognized SSL message: service:jmx:rmi:///jndi/rmi://our.host.com:10052/jmxrmi

I have increased the logging level of the gateway and this is appearing in the logs

2021-04-19 10:16:03.034 [pool-2-thread-4] DEBUG com.zabbix.gateway.SocketProcessor - starting to process incoming connection
2021-04-19 10:16:03.035 [pool-2-thread-4] DEBUG c.z.gateway.BinaryProtocolSpeaker - reading Zabbix protocol header
2021-04-19 10:16:03.035 [pool-2-thread-4] DEBUG c.z.gateway.BinaryProtocolSpeaker - reading 8 bytes of data length
2021-04-19 10:16:03.035 [pool-2-thread-4] DEBUG c.z.gateway.BinaryProtocolSpeaker - reading 191 bytes of request data
2021-04-19 10:16:03.035 [pool-2-thread-4] DEBUG c.z.gateway.BinaryProtocolSpeaker - received the following data in request: {"request":"java gateway jmx","jmx_endpoint":"service:jmx:rmi:///jndi/rmi://our.host.com:10052/jmxrmi","keys":["jmx[\"java.lang:type=GarbageCollector,name=ParNew\",Co llectionTime]"]}
2021-04-19 10:16:03.036 [pool-2-thread-4] DEBUG com.zabbix.gateway.SocketProcessor - RMI SSL hint cache cleanup is scheduled on 1618903451644, now is: 1618820163036
2021-04-19 10:16:03.036 [pool-2-thread-4] DEBUG com.zabbix.gateway.SocketProcessor - dispatched request to class com.zabbix.gateway.JMXItemChecker
2021-04-19 10:16:03.036 [pool-2-thread-4] DEBUG c.z.g.ZabbixJMXConnectorFactory - connecting to JMX agent at 'service:jmx:rmi:///jndi/rmi://our.host.com:10052/jmxrmi'
2021-04-19 10:16:03.039 [pool-2-thread-5] DEBUG com.zabbix.gateway.SocketProcessor - starting to process incoming connection
2021-04-19 10:16:03.039 [pool-2-thread-5] DEBUG c.z.gateway.BinaryProtocolSpeaker - reading Zabbix protocol header
2021-04-19 10:16:03.040 [pool-2-thread-5] WARN com.zabbix.gateway.SocketProcessor - error processing request: bad protocol header: 4A 52 4D 49 00
2021-04-19 10:16:03.040 [pool-2-thread-5] DEBUG com.zabbix.gateway.SocketProcessor - error caused by
com.zabbix.gateway.ZabbixException: bad protocol header: 4A 52 4D 49 00
at com.zabbix.gateway.BinaryProtocolSpeaker.getReques t(BinaryProtocolSpeaker.java:59)
at com.zabbix.gateway.SocketProcessor.run(SocketProce ssor.java:57)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
2021-04-19 10:16:03.040 [pool-2-thread-5] DEBUG c.z.gateway.BinaryProtocolSpeaker - sending the following data in response: {"response":"failed","error":"bad protocol header: 4A 52 4D 49 00"}
2021-04-19 10:16:03.041 [pool-2-thread-5] DEBUG com.zabbix.gateway.SocketProcessor - finished processing incoming connection
2021-04-19 10:16:03.041 [pool-2-thread-4] DEBUG c.z.g.ZabbixJMXConnectorFactory - connecting to JMX agent at 'service:jmx:rmi:///jndi/rmi://our.host.com:10052/jmxrmi'
2021-04-19 10:16:03.050 [pool-2-thread-2] DEBUG com.zabbix.gateway.SocketProcessor - starting to process incoming connection
2021-04-19 10:16:03.050 [pool-2-thread-2] DEBUG c.z.gateway.BinaryProtocolSpeaker - reading Zabbix protocol header
2021-04-19 10:16:03.060 [pool-2-thread-2] WARN com.zabbix.gateway.SocketProcessor - error processing request: bad protocol header: 16 03 03 01 94
2021-04-19 10:16:03.060 [pool-2-thread-2] DEBUG com.zabbix.gateway.SocketProcessor - error caused by
com.zabbix.gateway.ZabbixException: bad protocol header: 16 03 03 01 94
at com.zabbix.gateway.BinaryProtocolSpeaker.getReques t(BinaryProtocolSpeaker.java:59)
at com.zabbix.gateway.SocketProcessor.run(SocketProce ssor.java:57)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
2021-04-19 10:16:03.060 [pool-2-thread-2] DEBUG c.z.gateway.BinaryProtocolSpeaker - sending the following data in response: {"response":"failed","error":"bad protocol header: 16 03 03 01 94"}
2021-04-19 10:16:03.060 [pool-2-thread-2] DEBUG com.zabbix.gateway.SocketProcessor - finished processing incoming connection
2021-04-19 10:16:03.061 [pool-2-thread-4] WARN com.zabbix.gateway.SocketProcessor - error processing request, item "jmx["java.lang:type=GarbageCollector,name=ParNew",Coll ectionTime]" failed: Unsupported or unrecognized SSL message: service:jmx:rmi:///jndi/rmi://our.host.com:10052/jmxrmi
2021-04-19 10:16:03.062 [pool-2-thread-4] DEBUG com.zabbix.gateway.SocketProcessor - error caused by
com.zabbix.gateway.ZabbixException: Unsupported or unrecognized SSL message: service:jmx:rmi:///jndi/rmi://our.host.com:10052/jmxrmi
at com.zabbix.gateway.JMXItemChecker.getValues(JMXIte mChecker.java:168)
at com.zabbix.gateway.SocketProcessor.run(SocketProce ssor.java:81)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
2021-04-19 10:16:03.062 [pool-2-thread-4] DEBUG c.z.gateway.BinaryProtocolSpeaker - sending the following data in response: {"response":"failed","error":"Unsupported or unrecognized SSL message: service:jmx:rmi:\/\/\/jndi\/rmi:\/\/our.host.com:10052\/jmxrmi"}
2021-04-19 10:16:03.062 [pool-2-thread-4] DEBUG com.zabbix.gateway.SocketProcessor - finished processing incoming connection

The JMX interface is set up like this:

CATALINA_OPTS="-Dcom.sun.management.jmxremote ${CATALINA_OPTS}"
CATALINA_OPTS="-Dcom.sun.management.jmxremote.port=8099 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=xx.yy.zz.124 ${CATALINA_OPTS}"

This seems to be working as i can connect to xx.yy.zz.124:8099 without SSL using JConsole.

Im not sure if the problem appears between Zabbix Server and the Gateway or between the gateway and the Java application.

I haven't found any clues on
how the gateway knows how to find the Java appications JMX interface
if the gateway connects via PID or IP / Portnumber
where to configure those in the gateway
how to control if the communication between zabbix server and the gateway is encrypted
I would be okay with unencrypted connections

In the host configuration on the zabbix server if have tried the following settings under the encryption tab
No encryption = On, PSK = Off
No encryption = Off, PSK = On
No encryption = On, PSK = On
The PSK encryption was working fine between Zabbix Server and the zabbx agent.

Any help would be appreciated,
thanks in advance
Jens

[j.kisters]

I managed to solve this,
from reading the docs i assumed the connection must be made from the zabbix server to the zabbix java agent.
But that is not the case, zabbix server connects directly to the java application JMX port, the java gateway is not required.

[BigSmooth]

Port 10052 is for JGW, but it seems your tomcat JMX port is 8099.
Zabbix request it local JGW to get metrics from remote JMX.