Zabbix Agent Sends Wrong Value or Lastest Data Shows It Wrong

[anilkocaturk]

Hi All,

I have a Zabbix server and many zabbix agents on different locations (Except server, out of my network all agents, their all on active type, I can not check their values via 10050 port with passive agent mode)

I wrote a userparameter which give some numeric result for this clients(adding it on agent install phase). For example "ls -ltrh | wc -l". This questions result must be equal or greater than 0. I added this parameter as a item for exapmle question1. And added trigger for this item like "{template:question1.max(360m)}=0"means if folder empty for last 6 hours there is a problem.

It working on some clients correctly include if folder is empty showing problem or if folder is not empty like has 5 files items value coming 5 on lastest data dashboard.
But it's not working on some clients. For example the folder has 5 files but on lastest data dashboard it's value is 0. And problem is triggering normaly. When I tried userparameter's question manually (like I'm writing "ls -ltrh | wc -l" on to console) it show 5.

I'm really confused at this issue, it's working on some clients and not working correctly on others.

How can I detect where is the problem really. Is agent sending wrong value (0) or sending (5) but I'm receiving it as (0).

Thanks for all helps.

[dimir]

Firstly, you don't need to implement your custom item for counting number of files in a directory. There is Zabbix agent item for that vfs.dir.count .
As to your issue, I'd check agent/server (or proxy, if used) connectivity. Also check agent and server (or proxy, if used) log files for any errors.

[anilkocaturk]

Hi dimir

Firstly thanks for your reply.
Actually I'm not counting number of files in a directory, it was just an example. My original userparameter is "USERPARAMETER='log.question,tail -n 1000 /opt/xxxxxx/audit/xxxxxx.log |grep -a $(date +"%Y.%m.%d")|egrep "written" | wc -l'" and trigger is {template:log.question.max(360m)}=0
The result of "tail -n 1000 /opt/xxxxxx/audit/xxxxxx.log |grep -a $(date +"%Y.%m.%d")|egrep "written" | wc -l" is return 830 in client's shell. But in Lastest Data screen it seems 0. All other item's values are correct. I cant adding agent's log file and server's log file to attachments cause of size but I'm sharing a link of wetransfer for take a look https://we.tl/t-WfOJX85bgs . Agent working only active mode so means 10050 port is close.

About this problem my friend told me maybe client's perform cause of that. Agent trying send all values every 2 minutes at same time and client's resources not enough for send all at same time. For example log.question item's value at the end of line which values go to server. Cause of low resources it can not send the log.question's value (830) to server never. He's opinion is that.

[anilkocaturk]

up ++
Still waiting for some response.
Thanks for help.

[dimir]

Again, there is item log.count that does exactly the same thing, no need for user parameters. As to latest data, what is the timestamp of that 0? Is it up-to-date?
Make sure to set item's value type to "Numeric (unsigned)". See here on how to configure other item parameters of log.count .
P. S. the log file on wetransfer is deleted.

[Markku]

Hi, every time someone complains that a custom script does not work in Zabbix agent while working when run manually, the first question should be: Are you sure the user Zabbix agent runs on (like "zabbix") has sufficient access to do the things in the script? Have you tried running the script as "zabbix" user?

Markku

[anilkocaturk]

Hi dimir, I'll try log.count item. I'll feedback to you. Thanks for help. (Btw as to lastest data it's up-to-date and item's value type is "Numeric (unsigned)")

Hi Markku, yes agent has sufficient access. Other costum items work well.

[Markku]

Ok, if you are still experiencing the problems and you are unsure what data the agent sends, I would recommend capturing the agent communication with tcpdump or Wireshark, for at least a couple of sending intervals. Tcpdump can be used like

sudo tcpdump -s 0 -w capture.pcap port 10051

When looking at the capture file in Wireshark there are some dissectors if desired: https://github.com/markkuleinio/wire...bix-dissectors

Other way to troubleshoot the agent data would be saving all the data in a local file as well, like adding

 | tee -a /tmp/agentdata

in your shell script.

Markku

[anilkocaturk]

Thanks you Markku I'll try it and feedback to you. Thanks a lot for help

[Coyoteerex]

Ok, if you are still experiencing the problems and you are unsure what data the agent sends, I would recommend capturing the agent communication with tcpdump or Wireshark, for at least a couple of sending intervals. Tcpdump can be used like

sudo tcpdump -s 0 -w capture.pcap port 10051

When looking at the capture file in Wireshark there are some dissectors if desired: https://github.com/markkuleinio/wire...bix-dissectors/mycfavisit

Other way to troubleshoot the agent data would be saving all the data in a local file as well, like adding

 | tee -a /tmp/agentdata

in your shell script.

Markku

Thank you so much for sharing such a useful information. I will definitely share this with others.