Zabbix Agent is not accessible

[rumor]

Hi to everybody!
I have one issue though

I do have one HP-UX server being monitored. Everything work smooth and being ok, but every given night i do have {server_name:status.last(0)}=2 multiple times (for 0.8-2 min) during couple of hours. Last night all jobs at server's crontab (the only one) were stopped at all - but it didn't change a damn thing . So, seems, there aren't any kind of processes that could affect agent's work (proc_load or whatsoever is ok ).

I repeat: during the rest of the time everything is ok!

Log files during 2-hours night period:
Checking server's syslog (zabbix_agent logs too) - everything is ok!
Checking Main monitoring server's zabbix_suckerd log:

xxxxxx:xxxxxxxx:xxxxxx Got empty string from [server_name]. Parameter [procload or whatever]
xxxxxx:xxxxxxxx:xxxxxx Assuming that agent dropped connection because of access permissions
xxxxxx:xxxxxxxx:xxxxxx Host [server_name] will be checked after [60] seconds

I'll be glad to hear any ideas

[Alexei]

Looking at the log file (thanks for providing it!) I see that the agent was unable to answer ZABBIX server's query within Timeout (defined in zabbix_agentd.conf) seconds. The agent just drops connection in this case, that's why ZABBIX server thinks that no data received.

This probably happened due to very high load on the host, slow execution of an UserParameter (if you use any), or combination of 1 and 2.

How to get rid of this? I would suggest to:

1. check if you use UserParameters that may work slow. Optimise it!
2. check Timeout parameter for both ZABBIX server and the agent. Increase it for zabbix_agentd first if needed.

I hope this helps!

[rumor]

Thanks for help and speed! Classy!

I guess I just didn't describe it properly

So, sheme is:

Zabbix server (zabbix_suckerd) <-> HP-UX (running zabbix_agent under init.d daemon)

1. zabbix_agent.conf on HP-UX: I've checked UserParameters - nothing even close to smth gargantuan or something hevy.
2. As I understand it is not Timeout parameter on zabbix_agent.conf on HP-UX but Timeout parameter on zabbix_suckerd.conf on Zabbix should be changed. Right? But it's not an option anyway Right now on zabbix_suckerd.conf Timeout=3 - I guess it's pretty enough for all agents (at least for such not complex UserParameters as is at HP-UX: bdf, netstat, sar, uptime, ps - with variations)
3. As i mentioned before there is no very high load on the host (or even high, usually proc_load smth about 1)

So my question was: What else could be a factor for {server_name:status.last(0)}=2 ?

P.S. Sorry for possible lame
P.P.S. I just find at zabbix manual that init.d must contain
zabbix_agent stream tcp nowait.3600 zabbix /opt/zabbix/bin/zabbix_agent. Well, at my HP-UX init.d there is not such line I'm not familiar fith these params - what does it mean? Could it be the reason?

[Alexei]

Possible reasons

So, you're running zabbix_agent. The fact doesn't make any difference.

What really happens?

1. ZABBIX server sends a request to the host
2. inetd receives the request and launches zabbix_agent to process it
3. zabbix_agent processes it. If this is UserParameter then the agent executes required commands
4. In case of timeout, the agent just kills himself. No data sent back to the server.
5. If no erros occured, the agent sends required information back to the server.

What may go wrong?

1. inetd may reject connections because it detects flooding. Parameter nowait.3600 means that inetd may accept no more than 60 (3600/60, if I remember correctly) connections per second. By default it is less than that. I do not remeber how inetd reacts in this case. I think it just drops all incoming connections to the specific port.
2. Timeout. I believe that bdf makes sync, which may be very slow under certain disk I/O activity. Could be backup, copy of large files. So, the command may take more than 3 seconds. By the way, why you're using bdf?
3. UserParameter uses command which may fail (return nothing) under some conditions.

Hopefully it sheds some light.

[rumor]

Timeout. I believe that bdf makes sync, which may be very slow under certain disk I/O activity. Could be backup, copy of large files. So, the command may take more than 3 seconds. By the way, why you're using bdf?

First of all - I got monitoring system as a legacy from my ex-co-worker So, for now I am plannining (actually I am already there) go into it from the surface to the bottom and back.
I believe Zabbix's own diskfree[*] under HP-UX is not workinking correctly (Zabbix agent version is 1.0beta6 ).
bdf - probably couse it's easy to use awk to cut data. Which one is faster df -b or bdf - I just didn't ask myself, trully speaking!

P.S. Alexei if we do have official Zabbix support from You, should I use this forum or I can annoy you personally?

[Alexei]

I believe Zabbix's own diskfree[*] under HP-UX is not workinking correctly (Zabbix agent version is 1.0beta6 ).

I would suggest to upgrade it to the latest agent, 1.0. I have no information about any problems with the agent.

By the way, precompiled agents can be downloaded from http://www.zabbix.com/download.php

P.S. Alexei if we do have official Zabbix support from You, should I use this forum or I can annoy you personally?

The forum is supposed to be used for free support and as a knowledge base for all users of ZABBIX.

For commercial support, email me privately or give me a call. I'm sure you have my number

[charles]

The forum is supposed to be used for free support and as a knowledge base for all users of ZABBIX.

For commercial support, email me privately or give me a call. I'm sure you have my number

It would be nice if you did it here, so we could benefit from the answers though (and populate the knowledge base)

charles

[cooper]

thats not a bad idea

sorry to hijack this fourm, but . . . .

Alexi, maybe you could set up a fourm list that only people who had support contracts with you could post in. Then you could respond to those 1st, but the community would benefit from the FAQ like nature of this board. Just a thought. If you think its not in your or your clients best interest, i totally understand.

thanks

cooper

[charles]

Actually this is a very good idea! Alexei, use the power of these forums. You can make a private forum for paid support. Any threads that you wish to sahe can be moved to the other forums, or someone can manually distil the good info and put in on the FAQ or forums elsewhere.

charles

sorry to hijack this fourm, but . . . .

Alexi, maybe you could set up a fourm list that only people who had support contracts with you could post in. Then you could respond to those 1st, but the community would benefit from the FAQ like nature of this board. Just a thought. If you think its not in your or your clients best interest, i totally understand.

thanks

cooper

[Alexei]

sorry to hijack this fourm, but . . . .

Alexi, maybe you could set up a fourm list that only people who had support contracts with you could post in. Then you could respond to those 1st, but the community would benefit from the FAQ like nature of this board. Just a thought. If you think its not in your or your clients best interest, i totally understand.

I'm sure not all commercial users are interested in sharing information especially discussing details of setup, internal infrastructure, and problems.

So, probably such a forum is a good idea for entry-level support. I'm still learning the forum software, vBulletin. It provides lots of possibilities to configure user permissions, user rights, setup groups, make hidden forums, hidden posts, etc etc.

Perhaps a hidden forum per each customer plus one public forum for all customers is the best approach.

I'm open to discuss any ideas!

[charles]

I didn't think of the privacy between each other

But, I think a forum per customer would get hard to manage. I think you should use some helpdesk software for that. RT is OS/free, but I really like cerberus (cerberusweb.com) which is not free, but the source is available if you need to customize. You can respond to tickets via email, or the web.

charles

I'm sure not all commercial users are interested in sharing information especially discussing details of setup, internal infrastructure, and problems.

So, probably such a forum is a good idea for entry-level support. I'm still learning the forum software, vBulletin. It provides lots of possibilities to configure user permissions, user rights, setup groups, make hidden forums, hidden posts, etc etc.

Perhaps a hidden forum per each customer plus one public forum for all customers is the best approach.

I'm open to discuss any ideas!

[rumor]

I guess we have a little off-topic situation here
...

I would suggest to upgrade it to the latest agent, 1.0. I have no information about any problems with the agent.

By the way, precompiled agents can be downloaded from http://www.zabbix.com/download.php

It was good idea, thanks. I did replace yesterday old zabbix agent and right now Zabbix's diskfree[*] under HP-UX working very properly (I even used pre-compiled agent). Though I do still suffer from {server_name:status.last(0)}=2 (Right now I going to remove all (~10)UserParameters which are difened to use bdf. I hope it will change things for me, if not - we'll see whats up!).

[lcondado]

Commercial Support

Maybe you could respond first to users who had commercial support, but in public forums.

Regards.