zabbix proxy not retrieving all data

[Age42]

I have a working server->proxy setup that has been in production for almost 2 years, starting with zabbix 1.6.1. Last week I upgraded the proxy from 1.8.2 to 1.8.3, and since then, some items are not updating. I am able to manually retrieve data from the agent using zabbix_get, but the proxy doesn't seem to record anything.

Here is output for the itemid from the zabbix_proxy logs in full debug (just grepping for the itemid - looking through the logs I didn't see anything else related):

11499:20100928:084701.434 In calculate_item_nextcheck (4853,60,"",1285678021)
11499:20100928:084701.434 In zbx_binary_heap_insert() key[4853]
11499:20100928:084701.434 In zbx_hashmap_get() key[4853]
11499:20100928:084701.434 End of zbx_hashmap_get() key[4853] value[-1]
11499:20100928:084701.434 In zbx_hashmap_set() key[4853] value[13]
11499:20100928:084701.435 In zbx_hashmap_set() key[4853] value[28]
11499:20100928:084701.468 In zbx_hashmap_set() key[4853] value[57]
11499:20100928:084701.643 In zbx_hashmap_set() key[4853] value[115]
[4853,0,"public","interfaces.ifTable.ifEntry.ifInOc tets.1",161,10219,"proc.cpu[rsuser,torrid]",60,0,0,"","%",0,0,"",0,"","","1","",1626,0,"","D SN=<database source name>\\n user=<user name>\\n password=<password>\\n sql=<query>","",0,0,"","","",""],
update items set type=0,snmp_community='public',snmp_oid='interface s.ifTable.ifEntry.ifInOctets.1',snmp_port=161,host id=10219,key_='proc.cpu[rsuser,torrid]',delay=60,status=0,value_type=0,trapper_hosts='', units='%',multiplier=0,delta=0,snmpv3_securityname ='',snmpv3_securitylevel=0,snmpv3_authpassphrase=' ',snmpv3_privpassphrase='',formula='1',logtimefmt= '',templateid=1626,valuemapid=0,delay_flex='',para ms='DSN=<database source name>\\n user=<user name>\\n password=<password>\\n sql=<query>',ipmi_sensor='',data_type=0,authtype=0 ,username='',password='',publickey='',privatekey=' ' where itemid=4853;
11500:20100928:084702.613 In zbx_hashmap_set() key[4853] value[231]
11511:20100928:084737.484 In zbx_hashmap_set() key[4853] value[115]
11504:20100928:084739.657 In zbx_hashmap_set() key[4853] value[57]
11511:20100928:084743.704 In zbx_hashmap_set() key[4853] value[28]
11503:20100928:084752.038 In zbx_hashmap_set() key[4853] value[13]
11507:20100928:084753.079 In zbx_hashmap_set() key[4853] value[6]
11507:20100928:084753.080 In zbx_hashmap_set() key[4853] value[2]
11507:20100928:084753.081 In zbx_hashmap_set() key[4853] value[0]
11507:20100928:084753.082 In zbx_hashmap_remove() key[4853]

With full debug in the logs for the agent, there do not appear to be any checks coming in at all for this item.

All of the systems in question are running zabbix 1.8.3 (agents, proxies, server)

zabbix-agent Solaris Sparc
zabbix-proxy Solaris i386
zabbix-agent AMD64

We are losing valuable production data, any suggestions would be helpful.

Thanks

[_Phoenix_]

Confirm the problem 1.8.3 proxy

I installed 1.8.3 proxy.
zabbix-sender can't send eny data to it. In 1.8.2 worked fine

[vinny]

I am seeing the same behaviour as Age42.

I have several items not being monitored, whereas other items for the same host are working well.

* Looking at the agent : working well but no request made by the proxy for this item...
* Looking at the proxy : requesting manually this item works fine, but
in the debug log, no line about this item, nothing
I can see other items from this host being requested or not supported.

proxy_183=# select * from items where itemid = xxxxx;
     itemid      | type | snmp_community |                snmp_oid                 | snmp_port |     hostid      | description |
                              key_                                                | delay | history | trends | lastvalue | lastclock | prevvalue
| status | value_type | trapper_hosts | units | multiplier | delta | prevorgvalue | snmpv3_securityname | snmpv3_securitylevel | snmpv3_authpassp
hrase | snmpv3_privpassphrase | formula | error | lastlogsize | logtimefmt |   templateid    | valuemapid | delay_flex | params | ipmi_sensor | d
ata_type | authtype | username | password | publickey | privatekey | mtime
-----------------+------+----------------+-----------------------------------------+-----------+-----------------+-------------+-----------------
----------------------------------------------------------------------------------+-------+---------+--------+-----------+-----------+-----------
+--------+------------+---------------+-------+------------+-------+--------------+---------------------+----------------------+-----------------
------+-----------------------+---------+-------+-------------+------------+-----------------+------------+------------+--------+-------------+--
---------+----------+----------+----------+-----------+------------+-------
 xxxx |    0 | public         | interfaces.ifTable.ifEntry.ifInOctets.1 |       161 | xxxxx|             | system.run[sudo
/usr/toto |head -1] |  3600 |      90 |    365 |           |           |
|      0 |          1 |               |       |          0 |     0 |              |                     |                    0 |
      |                       | 1       |       |           0 |            | xxxxxx|          0 |            |        |             |
       0 |        0 |          |          |           |            |     0
(1 row)

vinny

[vinny]

I dont know why but some items in templates are not wokring anymore.

* So far, I only saw the behaviour with items of character data type.
* Deleting & re-creating the item solved the problem

Is this due to some change in the way charater item are treated ?

Vinny

[elvar]

I am being plagued with this as well!! I have roughly 150 hosts that I am pulling data from via snmpv3. For no reason that I can see tons of the hosts have stopped having data recorded. I can manually query fine but for 30 days+ I have no data. This is driving me nuts. I don't see anything wrong.

[DSon]

Suggestion..

Hi,

I've recently posted a thread regarding the fragility of zabbix proxy (v1.8.1) sending data to zabbix server (1.8.3). My issue surrounds the trigger evaluation logic.

It's possbily nothing to do with the problems you guys are experiencing, however try the following:

1) Disable ALL triggers on the zabbix server
2) Reboot ANY / ALL zabbix proxies
3) Observe the Latest Data screen to see if the item(s) in question starts to receive new data

I realise this is a drastic test to perform given that you (like me) have zabbix monitoring a production environment, however if you're desperate it could be worth a try.

Obviously once you have waited long enough to notice if this has helped, you can re-enable the triggers again.

Please let me know what the outcome is.
Danny.

NB. In my case, after point 1) I rebooted the zabbix server before rebooting the proxy servers, however it may have been coincidental so you may not have to