Hello Zabbix experts :-).
I am having a problem using jmx monitoring within zabbix to work with jboss EAP 6.4.8 and Zabbix server v2.2.14.
TLDR: How do I change the jmx connector from RMI to remote-jmx on a RPM install of zabbix?
The long version:
This tread had URL's of resources I found but i kept getting the error: Too many live links/images found in your post content. Please edit your post or contact the administrator. So I had to remove them. if you want the resources I can post them in the comments.
Here is what I have so far:
zabbix_java_gateway.conf:
LISTEN_IP="10.10.10.100"
LISTEN_PORT=10052
PID_FILE="/var/run/zabbixsrv/zabbix_java.pid"
START_POLLERS=5
TIMEOUT=3
zabbix_server.conf:
DebugLevel=4
JavaGateway=10.10.10.100
JavaGatewayPort=10052
StartJavaPollers=5
/usr/sbin/zabbix_java/lib/logback.xml
<root level="debug">
<appender-ref ref="FILE" />
</root>
For the time being I have disabled selinux and iptables.
From the jboss server I can telnet from both servers to both servers, so I know that the connections are not getting blocked by a firewall or security rules
Here is my configuration within the web console of zabbix:
I am pretty sure that I am making a connection to the Jboss server but the jboss does not allow RMI connections.
In these docs they state that "The JMX server listening in JBoss EAP 6.x is not an RMI Server"
I found a few resources that explain how to change your RMI jmx conector string to remote-jmx, BUT I am running a rpm install of the zabbix server.
My install:
yum install zabbix22-2.2.14-1.el6.x86_64
Since I am running a RPM install I can not for the life of me find the JMXItemChecker.java file so I can change the connection string:
OLD what I need to find:
url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + conn + ":" + port + "/jmxrmi");
What I need to change to:
url = new JMXServiceURL("service:jmx:remoting-jmx://" + conn + ":" + port);
I have done a find on my full file system first on the file name then for the string with in the file hoping it was just renamed in a newer release of zabbix.
Both of the find commands came back with no luck...
At this point the only thing I can think about doing is removing my install of zabbix and getting zabbix from source then compiling it my self.
I would hate to do that though because its nice to get zabbix updates with my monthly yumupdate's.
I would love to hear about any suggestions on this. I have posted my errors below:
Errors I am seeing:
After I turned on the debug mode in the zabbix_server.conf I see:
31256:20160928:105159.512 In substitute_key_macros() data:'jmx["java.lang:type=Memory","HeapMemoryUsage.used"]'
31256:20160928:105159.516 End of substitute_key_macros():SUCCEED data:'jmx["java.lang:type=Memory","HeapMemoryUsage.used"]'
31256:20160928:105159.580 In get_values_java() host:'ac0hlvmap18d' addr:'10.10.10.99' num:1
"request":"java gateway jmx",
"keys":["jmx[\"java.lang:type=Memory\",\"HeapMemoryUsage.use d\"]"]}]
31256:20160928:105159.611 JSON back [{"response":"failed","error":"java.io.IOExcepti on: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: non-JRMP server at remote endpoint]"}]
31256:20160928:105159.616 getting Java values failed: java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: non-JRMP server at remote endpoint]
31256:20160928:105159.623 End of get_values_java()
31256:20160928:105159.636 query [txnlev:1] [update hosts set jmx_disable_until=1475074379,jmx_error='java.io.IO Exception: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectI' where hostid=12528]
From the web console If I click on the host I see the below:
Which is the same error that I see in the debug logs
Thank you
I am having a problem using jmx monitoring within zabbix to work with jboss EAP 6.4.8 and Zabbix server v2.2.14.
TLDR: How do I change the jmx connector from RMI to remote-jmx on a RPM install of zabbix?
The long version:
This tread had URL's of resources I found but i kept getting the error: Too many live links/images found in your post content. Please edit your post or contact the administrator. So I had to remove them. if you want the resources I can post them in the comments.
Here is what I have so far:
zabbix_java_gateway.conf:
LISTEN_IP="10.10.10.100"
LISTEN_PORT=10052
PID_FILE="/var/run/zabbixsrv/zabbix_java.pid"
START_POLLERS=5
TIMEOUT=3
zabbix_server.conf:
DebugLevel=4
JavaGateway=10.10.10.100
JavaGatewayPort=10052
StartJavaPollers=5
/usr/sbin/zabbix_java/lib/logback.xml
<root level="debug">
<appender-ref ref="FILE" />
</root>
For the time being I have disabled selinux and iptables.
From the jboss server I can telnet from both servers to both servers, so I know that the connections are not getting blocked by a firewall or security rules
Here is my configuration within the web console of zabbix:
I am pretty sure that I am making a connection to the Jboss server but the jboss does not allow RMI connections.
In these docs they state that "The JMX server listening in JBoss EAP 6.x is not an RMI Server"
I found a few resources that explain how to change your RMI jmx conector string to remote-jmx, BUT I am running a rpm install of the zabbix server.
My install:
yum install zabbix22-2.2.14-1.el6.x86_64
Since I am running a RPM install I can not for the life of me find the JMXItemChecker.java file so I can change the connection string:
OLD what I need to find:
url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + conn + ":" + port + "/jmxrmi");
What I need to change to:
url = new JMXServiceURL("service:jmx:remoting-jmx://" + conn + ":" + port);
I have done a find on my full file system first on the file name then for the string with in the file hoping it was just renamed in a newer release of zabbix.
Both of the find commands came back with no luck...
At this point the only thing I can think about doing is removing my install of zabbix and getting zabbix from source then compiling it my self.
I would hate to do that though because its nice to get zabbix updates with my monthly yumupdate's.
I would love to hear about any suggestions on this. I have posted my errors below:
Errors I am seeing:
After I turned on the debug mode in the zabbix_server.conf I see:
31256:20160928:105159.512 In substitute_key_macros() data:'jmx["java.lang:type=Memory","HeapMemoryUsage.used"]'
31256:20160928:105159.516 End of substitute_key_macros():SUCCEED data:'jmx["java.lang:type=Memory","HeapMemoryUsage.used"]'
31256:20160928:105159.580 In get_values_java() host:'ac0hlvmap18d' addr:'10.10.10.99' num:1
"request":"java gateway jmx",
"keys":["jmx[\"java.lang:type=Memory\",\"HeapMemoryUsage.use d\"]"]}]
31256:20160928:105159.611 JSON back [{"response":"failed","error":"java.io.IOExcepti on: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: non-JRMP server at remote endpoint]"}]
31256:20160928:105159.616 getting Java values failed: java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: non-JRMP server at remote endpoint]
31256:20160928:105159.623 End of get_values_java()
31256:20160928:105159.636 query [txnlev:1] [update hosts set jmx_disable_until=1475074379,jmx_error='java.io.IO Exception: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectI' where hostid=12528]
From the web console If I click on the host I see the below:
Which is the same error that I see in the debug logs
Thank you