New item error after migrating to Postgres

[rhinowal]

I need your help.
I recently migrated a working Zabbix from Mysql to a new server with Postgres with TimescaleDB.
I followed these instructions here: https://assets.zabbix.com/files/even...2PgSQL-ENG.pdf

And there seemed to be no problems, server seemed to work fine all data was there and working.
However now I have a strange issue:
I have added a new item. This was done by cloning a fully working item. This cloned item has now several issues:
1. When I want to add it to a dashboard widget or enter the name into a filter, it does not get found / displayed. However when I click to show all items of a host, it is there and I can select it.
2. The item is working and under Monitoring -> Latest Data I can see the item and it has latest actual values. But when I go via Data Collection -> Hosts -> Select Host -> Item -> Latest data, I receive: No data found. When I go via Data Collection -> Hosts -> Select Host -> Item -> Graph, I see a fully working graph of that item.
3. I can add the item to a dashboard, I tried Top Hosts and Item Value, and it displays its values fine. But when I add the item to a graph, no values get plotted.

So it looks like that the item is present, but somehow not fully available. Like some internal database table dependencies are not 100% correct.

I have restarted Zabbix server and Postgres man times, this does not help. I also tries this solution: https://www.zabbix.com/forum/zabbix-...r-is-restarted but it did not solve my problem.

There is one thing that I wonder if this could be the reason: I think I have used the latest TimescaleDB and received a warning that version 2.14 is not supported. I have set Zabbix to allow unsupported. Could that be the reason for the failure and should I try to downgrade it to 2.13?

[cyber]

I have a gut feeling, it misses some data from DB... Maybe missing some internal triggers or something... It would be a handful of work, but you could compare PG scripts (those what are used to create DB out of the box) and what actually exists in your DB... IIRC, someone described similar issues with some major version upgrade, where hosts table did not populate name_upper, as it missed some triggers... I think there is similar field in items...

[cyber]

What version did you install?
TSDB 2.14 is supported for 6.4.13, 6.0.28...

But I doubt this is your issue... But what is your issue exactly, I do not know also..

[rhinowal]

What version did you install?
TSDB 2.14 is supported for 6.4.13, 6.0.28...

But I doubt this is your issue... But what is your issue exactly, I do not know also..

My Zabbix server is version 6.4.12. Maybe I shall update?
Yes, my issue is really really weird.

[markfree]

I'm not sure if this is the problem, but it can't hurt to check.

Make sure your cloned item has the correct "type of information". Numeric values set as character or text will show up in the "Item Value" or "Top Hosts" widgets, but not in graphs. Textual items are not found for graphs.

Before filtering items in "Latest data", reset the filter to make sure there is no cached data in the filter fields.

[rhinowal]

I'm not sure if this is the problem, but it can't hurt to check.

Make sure your cloned item has the correct "type of information". Numeric values set as character or text will show up in the "Item Value" or "Top Hosts" widgets, but not in graphs. Textual items are not found for graphs.

Before filtering items in "Latest data", reset the filter to make sure there is no cached data in the filter fields.

Thanks for your idea.
I have double checked and the cloned item has the same type of information set.
I even created a new item to check if maybe cloning of existing items is the cause. But it is not. The new item behaves the same strange way.

[markfree]

Can you illustrate your issue with pictures?

[rhinowal]

Can you illustrate your issue with pictures?

I can try.
So look: I pick an item, that is fully working and clone it.
The new item appears and I can change name and assign a new key. I chose TestItem as name.
When I save, I expect the new item to work exactly as the original I have cloned it from.

So I try to add it as graph to a dashboard. However when I type the name into the select field as item pattern, the name of the new item does not appear. When I click on select then I can scroll through all items and find the name of the new item. I can select it. But no graph appears.
Same goes for when I add the item with widget item value. I can not select it via item pattern only via Select and scroll. But this time (with the item value widget) I get values for the item.


So to me it appears that the cloned item is present in a way, but not full 'registered'. That's why the item pattern search and the graph cannot find it but the item value widget can see it and display its values.

[rhinowal]

Let me add some more description:
When I go to Monitoring -> Latest data -> Zabbix Server, I enter 'TestItem' as name, nothing gets found:


But the item is there, it even has values. When I do not enter the name:


And it has a graph, so it is very strange that there is no graph when I try with the graph widget:


So the item is basically fully functional, but only in theory. It does not show its values/graphs everywhere and cannot be searched for.

[rhinowal]

As an additional issue has surfaced (https://www.zabbix.com/forum/zabbix-...-after-restart) it seems that this Zabbix with Postgres is broken beyond repair. At least I could not trust any value because I would not know if it is correctly updated or not. So I guess I have to dump this and maybe retry.

[cyber]

Anything weird in PG logs?
For what its worth, you could upgrade to latest patchlevel... Would not screw up more, I suppose..

[rhinowal]

Anything weird in PG logs?
For what its worth, you could upgrade to latest patchlevel... Would not screw up more, I suppose..

No, I increased the log level but there is nothing that caught my attention and seemed abnormal. It appears to run perfectly normal.
I suspect an issue with the database. And to my knowledge minor upgrades don't do anything with the database. So it probably would not hurt but I don't know if it would help either.

[cyber]

I have a gut feeling, it misses some data from DB... Maybe missing some internal triggers or something... It would be a handful of work, but you could compare PG scripts (those what are used to create DB out of the box) and what actually exists in your DB... IIRC, someone described similar issues with some major version upgrade, where hosts table did not populate name_upper, as it missed some triggers... I think there is similar field in items...

[rhinowal]

I have a gut feeling, it misses some data from DB... Maybe missing some internal triggers or something... It would be a handful of work, but you could compare PG scripts (those what are used to create DB out of the box) and what actually exists in your DB... IIRC, someone described similar issues with some major version upgrade, where hosts table did not populate name_upper, as it missed some triggers... I think there is similar field in items...

Yes it feels like it is missing something that populates the new items throughout the database or sends events for new data.
As my new item did not appear immediately I had read and performed the solution mentioned here: https://www.zabbix.com/forum/zabbix-...801#post472801 and ran lines 2279 to 2844 against the database for the triggers ........

And when I wrote that I realized that I am a complete idiot. I ran the script against the wrong database, which I had switched from test to production.
I ran it again now and finally it seems that it is picking up all new items as it should.
OMG I can't thank you enough. I was about to nuke the new system switch back to the retired one losing a lot of previous data...... You have saved me from doing that and spending hours to rebuild it.
THANK YOU!!!!!!!

[cyber]

You are welcome...