"Problem name: No Instance(s) Available." on a trigger from a healthy hosts

[saibuster]

After reverting to a month old DB schema (Postgres), I started getting thousands of the following problem notifications:




The host is healthy and I can't see anything relevant on the server log.
The trigger for this problem is a python script, here is what I see on the agent log:

6316:20181018:024953.048 In process_active_checks() server:'zabbix.xxx.local' port:10051
6316:20181018:024953.048 Executing command 'c:\Python37\python.exe "c:\x\x.py"'
6316:20181018:024953.408 EXECUTE_STR() command:'c:\Python37\python.exe "c:\x\x.py"' len:0 cmd_result:''
6316:20181018:024953.408 for key [system.run[c:\Python37\python.exe "c:\x\x.py"]] received value []
6316:20181018:024953.408 In process_value() key:'xx-xx-xxxxx:system.run[c:\Python37\python.exe "c:\x\x.py"]' value:''
6316:20181018:024953.408 In send_buffer() host:'zabbix.xxx.local' port:10051 entries:0/100
6316:20181018:024953.408 End of send_buffer():SUCCEED
6316:20181018:024953.408 buffer: new element 0
6316:20181018:024953.408 End of process_value():SUCCEED
6316:20181018:024953.408 In need_meta_update() key:system.run[c:\Python37\python.exe "c:\x\x.py"]
6316:20181018:024953.408 End of need_meta_update():FAIL


The script works fine and had been working fine for a while. I started getting this message on multiple agents simultaneously.
Any idea what can cause this?

[saibuster]

Anyone? This issue is very frustrating and I can't find any mention of " No Instance(s) Available." on Zabbix manual/help.

[kloczek]

Just try to read what you wrote and ask yourself "do I see enough technical details to help this guy?"
No one here knows:
- what kind of metric is behind your trigger?
- what is your trigger definition and on what exact monitoring data operated this definition?
- why you have been reverting to the old db schema and why you've been fiddling db schema?
Try to add answers to those questions and look one more time on your issue.
You cannot find anything about " No Instance(s) Available." because this is part of you trigger definition probably in some template. Templates, items an triggers definitions are not immanent part of the zabbix. This is part of your monitoring configuration.

Your situation is like with application running on top of the OS kernel.
You are trying to find message which display some GUI application within documentation of the OS kernel. No, GUI is not part of the OS and you are on OS kernel forum.
If you want to have some help you need to provide descent amount details around the issue to understand other people with what you are dealing.

[saibuster]

Apologies for not sending the complete data needed, I will try to answer all your questions:

1) what kind of metric is behind your trigger? - This is a trigger that looks for dumps in a given directory, if the script returns any value than it is triggered.
Here are two examples of what it looks like when it returns nothing and returns a value: (not that this exact machine returns " No Instance(s) Available." for the script that we can clearly see that is working)






2) what is your trigger definition and on what exact monitoring data operated this definition?

Here is the trigger definition, it is used to send an email alert using the output string from the script (thus Name is defined as {ITEM.VALUE} )

Name: {ITEM.VALUE}
Expression: {SU-b5-WV32-6:system.run[c:\Python37\python.exe "c:\x\x.py"].strlen()}>0

3) Correction for my first post, I reverted to a month old Postgres VM Snapshot due to DB disk OOM that I was not able to correct (I was no longer able to login to the machine - I understand its a bad option but I had no other choice)

I see your point about "No Instance(s) Available.", the reason I thought it had something to do with Zabbix is due to the fact that my scripts never return such string.
So when {ITEM.VALUE} returned this message, I was under the impression that it was actually Zabbix agent or sever who sent it instead of my output. (See agent logs on debug level 5)

[kloczek]

So your script returns non-empty string which triggers alarm.
Probably because trigger definition has in "PROBLEM event generation mode" -> "multiple" on each evaluation of this string you have new alarm. If it would be "single" it will be created only one alarm on trigger result change (compare to prev evaluation)
Triggers are evaluated on the server only so looking on agent logs is pointless.

[saibuster]

It was actually already set to the configuration you have suggested. Unfortunately I overcame this issue by erasing and re-creating the problematic machines