java gateway crash

[pradhanparas]

Hi,

I have 7 servers that I monitor jvm via zabbix java gateway. But seems like the zabbix_java on the zabbix server crashes after around 24 hours and I have to restart to make it work again.

I changed START_POLLERS to 10 from the default value in setting.sh and StartPollers to 30 in zabbix_server.conf but same issue.

What am I missing?

Thanks!
Paras.

[heaje]

Have you looked in the java gateway's log file? What does it say in there? Is the JVM running out of memory?

[pradhanparas]

Hi,

Yes i did check the log. Mulitple warnings/error from java but I am not sure what do they mean.

Here is some part of the log file.

http://pastebin.com/tJuGJXd8


JVMs are fine are not running out of memory.

Thanks
Paras.

[pradhanparas]

Also,

what causes this?

31103:20130430:120145.511 JMX item [jmx["Catalina:type=GlobalRequestProcessor,name=htt p-8080",bytesReceived]] on host [kc5.domain] failed: another network error, wait for 15 seconds
31044:20130430:120159.015 resuming JMX checks on host [kc1.domain]: connection restored


Thanks
Paras.

[tchjts1]

Also,

what causes this?

Paras.

I was seeing a lot of that in my Zabbix server log for items in general. I increased the Timeout= value in zabbix_server.conf from 3 to 10, and that helped greatly.

[pradhanparas]

Thanks tchjts1. Looks like the Timeout increase to 10 fixed the issue.

Regarding zabbix_java crash, after I increase StartJavaPollers to 10 from 5, it stayed up for more than 24 hours but I will have to wait and see how it goes.

I still see the following message in the zabbix_java.log file. What is casuing it?

--
java.net.UnknownHostException: kc1.domain]
at com.zabbix.gateway.JMXItemChecker.getValues(JMXIte mChecker.java:100) ~[zabbix-java-gateway-2.0.2.jar:na]
at com.zabbix.gateway.SocketProcessor.run(SocketProce ssor.java:63) ~[zabbix-java-gateway-2.0.2.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker( ThreadPoolExecutor.java:1145) [na:1.7.0_19]
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) [na:1.7.0_19]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_19]
Caused by: java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ConfigurationException [Root exception is java.rmi.UnknownHostException: Unknown host: kc1.domain; nested exception is:
java.net.UnknownHostException: kc1.domain]
at javax.management.remote.rmi.RMIConnector.connect(R MIConnector.java:357) ~[na:1.7.0_19]
at javax.management.remote.JMXConnectorFactory.connec t(JMXConnectorFactory.java:267) ~[na:1.7.0_19]
at com.zabbix.gateway.JMXItemChecker.getValues(JMXIte mChecker.java:92) ~[zabbix-java-gateway-2.0.2.jar:na]
... 4 common frames omitted
--
Thanks!
Paras.

[heaje]

java.net.UnknownHostException: kc1.domain

The answer is in your question. The host "kc1.domain" doesn't appear to resolve to anything in DNS. Hence, it is an "Unknown Host".

[pradhanparas]

Well it is resolvable and in DNS and it works fine. Only occasionally I see that error but not always.

Thanks
Paras.