Ad Widget

**adriano** · 14-12-2011, 14:29

more info....

Logs from the master node showing node 4 (added just right now)...

Code:

[root@zabbix html]# tail -f /tmp/zabbix_server.log
  6895:20111214:102045.779 NODE 1: Received events from node 2 for node 2 datalen 510018
  6897:20111214:102046.229 NODE 1: Received history from node 2 for node 2 datalen 2587
  6897:20111214:102046.329 NODE 1: Received history_uint from node 2 for node 2 datalen 389
  6897:20111214:102054.921 NODE 1: Received events from node 2 for node 2 datalen 510018
  6896:20111214:102057.249 NODE 1: Received history from node 2 for node 2 datalen 2173
  6895:20111214:102057.358 NODE 1: Received history_uint from node 2 for node 2 datalen 892
  6896:20111214:102106.459 NODE 1: Received events from node 2 for node 2 datalen 510018
  6895:20111214:102106.951 NODE 1: Received history from node 2 for node 2 datalen 2581
  6897:20111214:102107.261 NODE 1: Received history_uint from node 2 for node 2 datalen 1152
  6897:20111214:102122.295 NODE 1: Received events from node 2 for node 2 datalen 510018
  6897:20111214:102130.638 NODE 1: Received configuration changes from slave node 4 for node 4 datalen 11606
  6897:20111214:102131.314 NODE 1: Sending configuration changes to slave node 4 for node 4 datalen 21899
  6895:20111214:102144.659 NODE 1: Received configuration changes from slave node 2 for node 2 datalen 8
  6895:20111214:102144.689 NODE 1: Sending configuration changes to slave node 2 for node 2 datalen 1671
  6895:20111214:102145.891 NODE 1: Received history from node 2 for node 2 datalen 7830
  6897:20111214:102146.071 NODE 1: Received history_uint from node 2 for node 2 datalen 2338

**adriano** · 14-12-2011, 14:35

logs from the just added slave node..

Code:

[root@manager ~]# tail -f /tmp/zabbix_server.log
  8187:20111214:101919.457 server #24 started [history syncer #4]
  8153:20111214:101919.457 server #6 started [poller #4]
  8189:20111214:101919.457 server #25 started [escalator #1]
  8192:20111214:101919.458 server #26 started [proxy poller #1]
  8194:20111214:101919.459 server #27 started [self-monitoring #1]
  8143:20111214:101919.459 server #0 started [main process]
  8156:20111214:101919.473 server #7 started [poller #5]
  8157:20111214:101919.476 server #8 started [unreachable poller #1]
  8180:20111214:101919.486 server #20 started [discoverer #1]
  8172:20111214:102023.426 Deleted 13737 records from history and trends
  8176:20111214:102023.708 NODE 4: Sending configuration changes to master node 1 for node 4 datalen 11606
  8176:20111214:102026.490 NODE 4: Received configuration changes from master node 1 for node 4 datalen 21899
  8176:20111214:102039.044 NODE 4: Sending alerts of node 4 to node 1 datalen 2939044
  8176:20111214:102054.734 NODE 4: Sending history_sync of node 4 to node 1 datalen 348734
  8176:20111214:102101.224 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303678
  8176:20111214:102106.913 NODE 4: Sending history_str_sync of node 4 to node 1 datalen 193360
  8176:20111214:102109.299 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102111.149 NOT OK
  8176:20111214:102111.352 NODE 4: Sending acknowledges of node 4 to node 1 datalen 4780
  8176:20111214:102112.014 NODE 4: Sending auditlog of node 4 to node 1 datalen 1779465
  8176:20111214:102120.699 NODE 4: Sending auditlog_details of node 4 to node 1 datalen 101494
  8176:20111214:102133.017 NODE 4: Sending configuration changes to master node 1 for node 4 datalen 4768
  8176:20111214:102135.224 NODE 4: Received configuration changes from master node 1 for node 4 datalen 7599
  8176:20111214:102146.282 NODE 4: Sending history_sync of node 4 to node 1 datalen 348750
  8176:20111214:102156.755 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303835
  8176:20111214:102206.862 NODE 4: Sending history_str_sync of node 4 to node 1 datalen 340
  8176:20111214:102209.501 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102216.964 NOT OK
  8176:20111214:102219.393 NODE 4: Sending history_sync of node 4 to node 1 datalen 348748
  8176:20111214:102225.485 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303572
  8176:20111214:102231.039 NODE 4: Sending history_str_sync of node 4 to node 1 datalen 104
  8176:20111214:102231.820 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102233.636 NOT OK
  8176:20111214:102234.702 NODE 4: Sending history_sync of node 4 to node 1 datalen 348657
  8176:20111214:102240.968 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303744
  8176:20111214:102248.123 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102250.142 NOT OK
  8176:20111214:102251.105 NODE 4: Sending history_sync of node 4 to node 1 datalen 348735
  8176:20111214:102257.834 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303610
  8176:20111214:102307.281 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102309.910 NOT OK
  8176:20111214:102311.018 NODE 4: Sending history_sync of node 4 to node 1 datalen 348795
  8176:20111214:102317.925 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303806
  8176:20111214:102325.759 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102328.372 NOT OK
  8176:20111214:102342.994 NODE 4: Sending configuration changes to master node 1 for node 4 datalen 8
  8176:20111214:102344.774 NODE 4: Received configuration changes from master node 1 for node 4 datalen 5379
  8176:20111214:102355.146 NODE 4: Sending history_sync of node 4 to node 1 datalen 348798
  8176:20111214:102406.839 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303647
  8176:20111214:102416.057 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102419.515 NOT OK
  8176:20111214:102420.906 NODE 4: Sending history_sync of node 4 to node 1 datalen 348762
  8176:20111214:102427.493 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303648
  8176:20111214:102436.639 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102439.086 NOT OK
  8176:20111214:102440.156 NODE 4: Sending history_sync of node 4 to node 1 datalen 348426
  8176:20111214:102447.989 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303819
  8176:20111214:102454.894 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102456.769 NOT OK
  8176:20111214:102457.755 NODE 4: Sending history_sync of node 4 to node 1 datalen 348557
  8176:20111214:102505.397 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303476
  8176:20111214:102513.487 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102515.483 NOT OK
  8176:20111214:102516.878 NODE 4: Sending history_sync of node 4 to node 1 datalen 348728
  8176:20111214:102524.309 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303734
  8176:20111214:102531.308 NODE 4: Sending history_str_sync of node 4 to node 1 datalen 302
  8176:20111214:102532.099 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102534.063 NOT OK
  8176:20111214:102549.628 NODE 4: Sending configuration changes to master node 1 for node 4 datalen 358
  8176:20111214:102551.408 NODE 4: Received configuration changes from master node 1 for node 4 datalen 5499
  8176:20111214:102600.517 NODE 4: Sending history_sync of node 4 to node 1 datalen 348783
  8176:20111214:102609.843 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303763
  8176:20111214:102616.730 NODE 4: Sending history_str_sync of node 4 to node 1 datalen 278
  8176:20111214:102617.591 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102619.483 NOT OK
  8176:20111214:102620.801 NODE 4: Sending history_sync of node 4 to node 1 datalen 348706
  8176:20111214:102628.097 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303621
  8176:20111214:102641.650 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102645.945 NOT OK
  8176:20111214:102647.841 NODE 4: Sending history_sync of node 4 to node 1 datalen 348708
  8176:20111214:102657.889 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303689
  8176:20111214:102708.828 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102713.526 NOT OK
  8176:20111214:102715.621 NODE 4: Sending history_sync of node 4 to node 1 datalen 348643
  8176:20111214:102723.206 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303746
  8176:20111214:102731.249 NODE 4: Sending history_str_sync of node 4 to node 1 datalen 72
  8176:20111214:102733.234 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102739.884 NOT OK
  8176:20111214:102759.831 NODE 4: Sending configuration changes to master node 1 for node 4 datalen 8
  8176:20111214:102802.298 NODE 4: Received configuration changes from master node 1 for node 4 datalen 5379
  8176:20111214:102816.654 NODE 4: Sending history_sync of node 4 to node 1 datalen 348666
  8176:20111214:102829.155 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303586
  8176:20111214:102838.215 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102842.821 NOT OK
  8176:20111214:102843.899 NODE 4: Sending history_sync of node 4 to node 1 datalen 348650
  8176:20111214:102851.718 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303614
  8176:20111214:102859.445 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102901.354 NOT OK
  8176:20111214:102902.390 NODE 4: Sending history_sync of node 4 to node 1 datalen 348758
  8176:20111214:102909.947 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303820
  8176:20111214:102917.047 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102918.920 NOT OK
  8176:20111214:102919.917 NODE 4: Sending history_sync of node 4 to node 1 datalen 348547
  8176:20111214:102927.430 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303597
  8176:20111214:102935.093 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102937.580 NOT OK
  8176:20111214:102938.754 NODE 4: Sending history_sync of node 4 to node 1 datalen 348679
  8176:20111214:102946.265 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303595
  8176:20111214:102954.525 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:102956.482 NOT OK
  8176:20111214:103011.251 NODE 4: Sending configuration changes to master node 1 for node 4 datalen 8
  8176:20111214:103013.216 NODE 4: Received configuration changes from master node 1 for node 4 datalen 5379
  8176:20111214:103019.175 NODE 4: Sending history_sync of node 4 to node 1 datalen 348236
  8176:20111214:103026.727 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303846
  8176:20111214:103034.381 NODE 4: Sending history_str_sync of node 4 to node 1 datalen 198
  8176:20111214:103038.835 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:103040.688 NOT OK
  8176:20111214:103041.636 NODE 4: Sending history_sync of node 4 to node 1 datalen 347840
  8176:20111214:103049.621 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303570
  8176:20111214:103057.658 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:103059.629 NOT OK
  8176:20111214:103100.714 NODE 4: Sending history_sync of node 4 to node 1 datalen 347771
  8176:20111214:103108.031 NODE 4: Sending history_uint_sync of node 4 to node 1 datalen 303674
  8176:20111214:103115.854 NODE 4: Sending events of node 4 to node 1 datalen 375037
  8176:20111214:103117.742 NOT OK
  8176:20111214:103118.781 NODE 4: Sending history_sync of node 4 to node 1 datalen 347821

**adriano** · 14-12-2011, 19:48

nothing yet..

The Master Node has been formatted and reinstalled.

We've done a clean install, without restoring the backuped data (corrupted data), but the Master Node still doesn't sync with the Slave Nodes.

We already tried:

- Change the NodeID of the Master;
- Change to the another IP address of the Master Node (external IP address);
- Format the Master Node;
- Update Zabbix to 1.8.9;
- Downgrade to 1.8.8 again;
- Delete all data from Master Node and start from scratch;

No solutions found yet..

Any idea?

**adriano** · 20-12-2011, 22:11

still nothing...

Sorry about this post, its not organized.

I tried to take some help in the support system, because I though it was a issue, but Oleksiy Zagorskyi said its not, so here we go....

------------------------

2011 Dec 16
Everything was working until we tried to update the Master Node from 1.8.8 to 1.8.9.

Here I have some info that I've already collected about everything:

We’ll be back soon!

http://www.zabbix.com/forum/showthread.php?t=24367

I already had updated Zabbix before with no problems, but this time the server get out of space because of InnoDB and I've lost all Master Node data.
SQL backup was corrupted, I've lost everything on the Master Node database.

There's no problem because didn't affected the slaves and the master isn't so important, but now it doesn't sync anything.

Please, I'm in a hurry, my employers gave me one week to fix it.

I can give any information you need, just ask.

Thank you.

Here are some logs from the slave (filtered):

1358:20111216:165534.090 Query [txnlev:0] [select masterid from nodes where nodeid=2]
1358:20111216:165534.090 Query [txnlev:0] [select masterid from nodes where nodeid=1]
1358:20111216:165534.090 Query [txnlev:0] [select ip,port from nodes where nodeid=1]
1358:20111216:165534.139 NODE 2: Sending [ZBX_GET_HISTORY_LAST_ID*2*2
alerts*alertid] to Node [1]
1358:20111216:165534.175 NODE 2: Receiving [200000000012160] from Node [1]
1358:20111216:165534.175 Query [txnlev:1] [begin;]
1358:20111216:165534.175 Query [txnlev:1] [commit;]
1358:20111216:165534.175 Query [txnlev:1] [begin;]
1358:20111216:165534.175 Query [txnlev:1] [select id,itemid,clock,value from history_sync where nodeid=2 order by id limit 10000]
1358:20111216:165534.176 NODE 2: Sending history_sync of node 2 to node 1 datalen 2396
1358:20111216:165534.176 Query [txnlev:1] [select ip,port from nodes where nodeid=1]
1358:20111216:165534.196 NODE 2: Sending [History*2*2*history_sync
1358:20111216:165534.298 NODE 2: Receiving [OK] from Node [1]
1358:20111216:165534.298 OK
1358:20111216:165534.298 Query [txnlev:1] [delete from history_sync where nodeid=2 and id<=44509441]
1358:20111216:165534.299 Query [txnlev:1] [commit;]
1358:20111216:165534.306 Query [txnlev:1] [begin;]
1358:20111216:165534.306 Query [txnlev:1] [select id,itemid,clock,value from history_uint_sync where nodeid=2 order by id limit 10000]
1358:20111216:165534.307 NODE 2: Sending history_uint_sync of node 2 to node 1 datalen 480
1358:20111216:165534.307 Query [txnlev:1] [select ip,port from nodes where nodeid=1]
1358:20111216:165534.324 NODE 2: Sending [History*2*2*history_uint_sync
1330:20111216:165534.335 Get value from agent result: '659947.522052'
1334:20111216:165534.337 Sending [vfs.fs.size[/,free]
]
1334:20111216:165534.338 Get value from agent result: '11515592704'

This time I do have some "OK"s in the log, but still nothing on the Master Node.

The imagem in attachment shows what I say: even in fresh install the data doesn't sync.

And the master only answer this:

32322:20111216:175930.853 NODE 1: Received events from node 2 for node 2 datalen 510018
32323:20111216:175932.235 NODE 1: Received history from node 2 for node 2 datalen 2037
32321:20111216:175932.339 NODE 1: Received history_uint from node 2 for node 2 datalen 657
32322:20111216:175940.974 NODE 1: Received events from node 2 for node 2 datalen 510018
32322:20111216:175942.376 NODE 1: Received history from node 2 for node 2 datalen 1890
32321:20111216:175942.463 NODE 1: Received history_uint from node 2 for node 2 datalen 213
32322:20111216:175951.091 NODE 1: Received events from node 2 for node 2 datalen 510018
32321:20111216:175952.649 NODE 1: Received history from node 2 for node 2 datalen 1699
32323:20111216:175952.736 NODE 1: Received history_uint from node 2 for node 2 datalen 449
32322:20111216:180001.297 NODE 1: Received events from node 2 for node 2 datalen 510018
32321:20111216:180002.680 NODE 1: Received history from node 2 for node 2 datalen 2152
32323:20111216:180002.788 NODE 1: Received history_uint from node 2 for node 2 datalen 700

In one of the re-installation tries we did get 7 hosts been added, but we've only got NULL on mysql our "Agent droped connection..... ZBX_TCP_READ..." ...

--------------------

2011 Dec 18 21:29
News:
Yesterday installed new hardware for testing on a fresh NEW NODE.

Today installed all software, including Zabbix Server Slave Node.

Everything worked fine, so the problem is with the "old" Slave Nodes, that doesn't sync anyway.

There is a problem switching the Master Node?

I'll be testing in the next days, and I'll post everything new that I get.

--------------------

2011 Dec 20 08:35
There's no network problems, and the workaround on ZBX-3996 didn't worked, tried excluding all entries with the script (what took about half of a day) but nothing new on the master node frontend, the data still doesn't sync.

--------------------

2011 Dec 20 10:00
MySQL data is being sent and received with no failed queries, but there's a lot of "rollback" entries in the Zabbix Master Node /var/log/mysql.log.

And some NULL info coming from the Slaves too, here are the logs:

28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'sysmaps_elements',selementid,1,concat_ws(',',s ysmapid,elementid,elementtype,iconid_off,iconid_on ,iconid_unknown,md5(label),case when label_location is null then 'NULL' else cast(label_location as char) end,x,y,md5(url),iconid_disabled,iconid_maintenanc e) from sysmaps_elements where 1=1 and selementid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'sysmaps',sysmapid,1,concat_ws(',',md5(name),wi dth,height,backgroundid,label_type,label_location, highlight) from sysmaps where 1=1 and sysmapid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'triggers',triggerid,1,concat_ws(',',md5(expres sion),md5(description),md5(url),status,value,prior ity,lastchange,dep_level,md5(comments),md5(error), templateid,type) from triggers where 1=1 and triggerid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'trigger_depends',triggerdepid,1,concat_ws(',', triggerid_down,triggerid_up) from trigger_depends where 1=1 and triggerdepid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'users',userid,1,concat_ws(',',md5(alias),md5(n ame),md5(surname),md5(passwd),md5(url),autologin,a utologout,md5(lang),refresh,type,md5(theme),attemp t_failed,md5(attempt_ip),attempt_clock,rows_per_pa ge) from users where 1=1 and userid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'usrgrp',usrgrpid,1,concat_ws(',',md5(name),gui _access,users_status,api_access,debug_mode) from usrgrp where 1=1 and usrgrpid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'users_groups',id,1,concat_ws(',',usrgrpid,user id) from users_groups where 1=1 and id between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'valuemaps',valuemapid,1,concat_ws(',',md5(name )) from valuemaps where 1=1 and valuemapid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'maintenances',maintenanceid,1,concat_ws(',',md 5(name),maintenance_type,md5(description),active_s ince,active_till) from maintenances where 1=1 and maintenanceid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'maintenances_hosts',maintenance_hostid,1,conca t_ws(',',maintenanceid,hostid) from maintenances_hosts where 1=1 and maintenance_hostid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'maintenances_groups',maintenance_groupid,1,con cat_ws(',',maintenanceid,groupid) from maintenances_groups where 1=1 and maintenance_groupid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'maintenances_windows',maintenance_timeperiodid ,1,concat_ws(',',maintenanceid,timeperiodid) from maintenances_windows where 1=1 and maintenance_timeperiodid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'timeperiods',timeperiodid,1,concat_ws(',',time period_type,every,month,dayofweek,day,start_time,p eriod,start_date) from timeperiods where 1=1 and timeperiodid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'regexps',regexpid,1,concat_ws(',',md5(name),md 5(test_string)) from regexps where 1=1 and regexpid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'user_history',userhistoryid,1,concat_ws(',',us erid,md5(title1),md5(url1),md5(title2),md5(url2),m d5(title3),md5(url3),md5(title4),md5(url4),md5(tit le5),md5(url5)) from user_history where 1=1 and userhistoryid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'expressions',expressionid,1,concat_ws(',',rege xpid,md5(expression),expression_type,md5(exp_delim iter),case_sensitive) from expressions where 1=1 and expressionid between 9800000000000000 and 9899999999999999
28230 Query insert into node_cksum (nodeid,tablename,recordid,cksumtype,cksum) select 98,'autoreg_host',autoreg_hostid,1,concat_ws(',',p roxy_hostid,md5(host)) from autoreg_host where 1=1 and autoreg_hostid between 9800000000000000 and 9899999999999999
28230 Query select curr.tablename,curr.recordid,prev.cksum,curr.cksum ,prev.sync from node_cksum curr, node_cksum prev where curr.nodeid=98 and prev.nodeid=curr.nodeid and curr.tablename=prev.tablename and curr.recordid=prev.recordid and curr.cksumtype=1 and prev.cksumtype=0 union all select curr.tablename,curr.recordid,prev.cksum,curr.cksum ,curr.sync from node_cksum curr left join node_cksum prev on prev.nodeid=curr.nodeid and prev.tablename=curr.tablename and prev.recordid=curr.recordid and prev.cksumtype=0 where curr.nodeid=98 and curr.cksumtype=1 and prev.tablename is null union all select prev.tablename,prev.recordid,prev.cksum,curr.cksum ,prev.sync from node_cksum prev left join node_cksum curr on curr.nodeid=prev.nodeid and curr.tablename=prev.tablename and curr.recordid=prev.recordid and curr.cksumtype=1 where prev.nodeid=98 and prev.cksumtype=0 and curr.tablename is null
28220 Query begin
28220 Query select nodeid from nodes where nodeid=2 and masterid=1
28220 Query select nodeid from nodes where masterid=1
28220 Query select description,expression,priority,type from triggers where triggerid=200200000012769
28220 Query rollback
111220 9:51:38 28224 Query select nodeid from nodes where nodeid=2 and masterid=1
28224 Query select nodeid from nodes where masterid=1
28224 Query select max(acknowledgeid) from acknowledges where 1=1 and acknowledgeid between 200000000000000 and 299999999999999
28224 Query select nodeid from nodes where nodeid=2 and masterid=1
28224 Query select nodeid from nodes where masterid=1
28224 Query select max(auditid) from auditlog where 1=1 and auditid between 200000000000000 and 299999999999999
28217 Query select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,mt.mediatypeid,mt.type,mt.description, mt.smtp_server,mt.smtp_helo,mt.smtp_email,mt.exec_ path,mt.gsm_modem,mt.username,mt.passwd,a.retries from alerts a,media_type mt where a.status=0 and a.mediatypeid=mt.mediatypeid and a.alerttype=0 and mt.mediatypeid between 100000000000000 and 199999999999999 order by a.clock
28224 Query select nodeid from nodes where nodeid=2 and masterid=1
28224 Query select nodeid from nodes where masterid=1
28224 Query select max(auditdetailid) from auditlog_details where 1=1 and auditdetailid between 200000000000000 and 299999999999999
111220 9:51:39 28224 Query select nodeid from nodes where nodeid=2 and masterid=1
28224 Query select nodeid from nodes where masterid=1
28224 Query select max(servicealarmid) from service_alarms where 1=1 and servicealarmid between 200000000000000 and 299999999999999
28224 Query select nodeid from nodes where nodeid=2 and masterid=1
28224 Query select nodeid from nodes where masterid=1
28224 Query select max(alertid) from alerts where 1=1 and alertid between 200000000000000 and 299999999999999
28219 Query select t.httptestid,t.name,t.applicationid,t.nextcheck,t. status,t.macros,t.agent,t.authentication,t.http_us er,t.http_password from httptest t,applications a,hosts h where t.applicationid=a.applicationid and a.hostid=h.hostid and t.nextcheck<=1324381899 and mod(t.httptestid,2)=0 and t.status=0 and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0) and t.httptestid between 100000000000000 and 199999999999999
28219 Query select min(t.nextcheck) from httptest t,applications a,hosts h where t.applicationid=a.applicationid and a.hostid=h.hostid and mod(t.httptestid,2)=0 and t.status=0 and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0) and t.httptestid between 100000000000000 and 199999999999999
28229 Query select t.httptestid,t.name,t.applicationid,t.nextcheck,t. status,t.macros,t.agent,t.authentication,t.http_us er,t.http_password from httptest t,applications a,hosts h where t.applicationid=a.applicationid and a.hostid=h.hostid and t.nextcheck<=1324381899 and mod(t.httptestid,2)=1 and t.status=0 and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0) and t.httptestid between 100000000000000 and 199999999999999
28229 Query select min(t.nextcheck) from httptest t,applications a,hosts h where t.applicationid=a.applicationid and a.hostid=h.hostid and mod(t.httptestid,2)=1 and t.status=0 and h.status=0 and (h.maintenance_status=0 or h.maintenance_type=0) and t.httptestid between 100000000000000 and 199999999999999
28233 Query select escalationid,actionid,triggerid,eventid,r_eventid, esc_step,status from escalations where status in (0,4,5,1) and nextcheck<=1324381899 and escalationid between 100000000000000 and 199999999999999
28224 Query begin
28224 Query select nodeid from nodes where nodeid=2 and masterid=1
28224 Query select nodeid from nodes where masterid=1
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022402 and clock=1324378800
28224 Query update trends set num=320,value_min=0.187500,value_avg=78882.777537, value_max=1878677.921300 where itemid=200200000022402 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000018472 and clock=1324378800
28224 Query update trends set num=320,value_min=0.317000,value_avg=0.607357,valu e_max=2.301200 where itemid=200200000018472 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022194 and clock=1324378800
28224 Query update trends set num=623,value_min=0.000000,value_avg=0.068419,valu e_max=1.680000 where itemid=200200000022194 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022399 and clock=1324378800
28224 Query update trends set num=623,value_min=0.000000,value_avg=1577.410403,v alue_max=105568.600000 where itemid=200200000022399 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022169 and clock=1324378800
28224 Query update trends set num=623,value_min=412.142900,value_avg=2027.085197 ,value_max=299669.400000 where itemid=200200000022169 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000018444 and clock=1324378800
28224 Query update trends set num=623,value_min=1071.400000,value_avg=53987.1845 70,value_max=96931.750000 where itemid=200200000018444 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000018474 and clock=1324378800
28224 Query update trends set num=312,value_min=0.025000,value_avg=0.085362,valu e_max=0.407300 where itemid=200200000018474 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000018440 and clock=1324378800
28224 Query update trends set num=624,value_min=940.000000,value_avg=6915.664107 ,value_max=415218.000000 where itemid=200200000018440 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022275 and clock=1324378800
28224 Query update trends set num=156,value_min=0.016700,value_avg=0.040667,valu e_max=0.068900 where itemid=200200000022275 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022195 and clock=1324378800
28224 Query update trends set num=312,value_min=0.000000,value_avg=0.043179,valu e_max=0.440000 where itemid=200200000022195 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000018445 and clock=1324378800
28224 Query update trends set num=624,value_min=761.600000,value_avg=2006.698031 ,value_max=24895.400000 where itemid=200200000018445 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022196 and clock=1324378800
28224 Query update trends set num=315,value_min=88.984300,value_avg=98.065355,va lue_max=99.864600 where itemid=200200000022196 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022217 and clock=1324378800
28224 Query update trends set num=104,value_min=99.283200,value_avg=99.283600,va lue_max=99.283700 where itemid=200200000022217 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000018441 and clock=1324378800
28224 Query update trends set num=625,value_min=0.000000,value_avg=4311.309796,v alue_max=48440.400000 where itemid=200200000018441 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022197 and clock=1324378800
28224 Query update trends set num=313,value_min=0.000000,value_avg=0.043143,valu e_max=0.257700 where itemid=200200000022197 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022276 and clock=1324378800
28224 Query update trends set num=632,value_min=0.000000,value_avg=0.038891,valu e_max=0.183300 where itemid=200200000022276 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022277 and clock=1324378800
28224 Query update trends set num=312,value_min=0.010000,value_avg=0.038871,valu e_max=0.093300 where itemid=200200000022277 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022171 and clock=1324378800
28224 Query update trends set num=624,value_min=0.000000,value_avg=138.591570,va lue_max=3990.400000 where itemid=200200000022171 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000018442 and clock=1324378800
28224 Query update trends set num=625,value_min=985.750000,value_avg=1994.110947 ,value_max=22449.000000 where itemid=200200000018442 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022166 and clock=1324378800
28224 Query update trends set num=625,value_min=400.000000,value_avg=2751.834463 ,value_max=64042.500000 where itemid=200200000022166 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022402 and clock=1324378800
28224 Query update trends set num=321,value_min=0.187500,value_avg=78684.044589, value_max=1878677.921300 where itemid=200200000022402 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000022168 and clock=1324378800
28224 Query update trends set num=624,value_min=0.000000,value_avg=138.464146,va lue_max=3452.666700 where itemid=200200000022168 and clock=1324378800
28224 Query select num,value_min,value_avg,value_max from trends where itemid=200200000018443 and clock=1324378800

----------------------

2011 Dec 20 10:13
Here is some info I've got from mysql:

In the master node I do have two slave nodes: one is the client node (nodeid 2), and the other is the test node (nodeid 98).
The test node was just a to verify if the Master Node was working.

Master Node info:
+--------+----------------+----------+----------------+-------+---------------+--------------+----------+----------+
| nodeid | name | timezone | ip | port | slave_history | slave_trends | nodetype | masterid |
+--------+----------------+----------+----------------+-------+---------------+--------------+----------+----------+
| 1 | Master | -3 | 127.0.0.1 | 10051 | 30 | 365 | 1 | 0 |
| 2 | Slave | -3 | zzz.zzz.zzz.zzz | 10051 | 90 | 365 | 0 | 1 |
| 98 | Test Slave | -3 | yyy.yyy.yyy.yyy | 10051 | 90 | 365 | 0 | 1 |
+--------+----------------+----------+----------------+-------+---------------+--------------+----------+----------+
3 rows in set (0.00 sec)

Slave Node info:
+--------+---------+----------+---------------+-------+---------------+--------------+----------+----------+
| nodeid | name | timezone | ip | port | slave_history | slave_trends | nodetype | masterid |
+--------+---------+----------+---------------+-------+---------------+--------------+----------+----------+
| 1 | Master | -3 | xxx.xxx.xxx.xxx | 10051 | 90 | 365 | 0 | 0 |
| 2 | Slave | -3 | 127.0.0.1 | 10051 | 180 | 365 | 1 | 1 |
+--------+---------+----------+---------------+-------+---------------+--------------+----------+----------+
2 rows in set (0.00 sec)

-------------------------

2011 Dec 20 16:09 - edited
Done, it's a bug... I tried to reproduce the error and could not sync the data from the test slave node with the master node server.

Testing steps:

Ok, my configurations are simple: two different networks in the internet, just added rules in the board firewalls to allow zabbix traffic from both sides.

After installing and converting the new Test Slave Node, I've enabled zabbix_agentd on it and generated a lot of data (about 50 items updating at 1 sec rate).

Waited for a while and then started playing.. I just let the slave node running until Master Node get enough data and then formatted the Master Node computer.

Installed new CentOS 6.0 on the master node computer configured all the network stuff.

Here is the installation steps:

Removed security stuff (not needed now)

yum erase selinux-policy
iptables -F
chkconfig --level 12345 iptables off

After disabling the firewall added repos:

wget http://pkgs.repoforge.org/rpmforge-r....rf.x86_64.rpm &&
rpm -ivh rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm &&
rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt &&
rpm -K rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm

wget http://download.fedora.redhat.com/pu...6-5.noarch.rpm &&
rpm -ivh epel-release-6-5.noarch.rpm &&
rpm --import http://download.fedora.redhat.com/pu...GPG-KEY-EPEL-6 &&
rpm -K epel-release-6-5.noarch.rpm

Installed dependencies:

yum install -y httpd gcc mysql mysql-server mysql-devel net-snmp net-snmp-devel net-snmp-utils net-snmp-libs curl curl-devel php php-mysql php-gd php-ldap php-bcmath php-common php-mbstring php-dom php-xml

Updated everything else:

yum upgrade -y
yum update -y

Started configuration:

/etc/init.d/mysqld start
/etc/init.d/httpd start

adduser zabbix

cd /home/zabbix &&
wget http://prdownloads.sourceforge.net/z...x-1.8.9.tar.gz &&
tar xzvf zabbix-1.8.9.tar.gz

vim /etc/my.cnf

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
innodb_file_per_table=1

mysql_secure_installation (default options)

mysql -p
SET PASSWORD FOR 'root'@'localhost' = PASSWORD('rootpassword');
create database zabbix character set utf8;
CREATE USER 'zabbix'@'localhost' IDENTIFIED BY 'zabbixpassword';
GRANT ALL PRIVILEGES ON zabbix.* TO 'zabbix'@'localhost' WITH GRANT OPTION;
quit;

cat /home/zabbix/zabbix-1.8.9/create/schema/mysql.sql | mysql zabbix -p &&
cat /home/zabbix/zabbix-1.8.9/create/data/data.sql | mysql zabbix -p &&
cat /home/zabbix/zabbix-1.8.9/create/data/images_mysql.sql | mysql zabbix -p

cd /home/zabbix/zabbix-1.8.9/ &&
./configure --enable-server --with-mysql --with-net-snmp --with-libcurl --enable-agent &&
make && make install

mkdir /etc/zabbix &&
cp /home/zabbix/zabbix-1.8.9/misc/conf/zabbix_server.conf /etc/zabbix/. &&
cp /home/zabbix/zabbix-1.8.9/misc/conf/zabbix_agentd.conf /etc/zabbix/. &&
cp /etc/zabbix/zabbix_server.conf /etc/zabbix/zabbix_server.conf.default &&
cp /etc/zabbix/zabbix_server.conf /etc/zabbix/zabbix_agentd.conf.default &&
chown -R zabbix:zabbix /etc/zabbix/

echo "NodeID=1" > /etc/zabbix/zabbix_server.conf &&
echo "LogFileSize=1" >> /etc/zabbix/zabbix_server.conf &&
echo "LogFile=/tmp/zabbix_server.log" >> /etc/zabbix/zabbix_agentd.conf &&
echo "DebugLevel=3" >> /etc/zabbix/zabbix_server.conf &&
echo "DBName=zabbix" >> /etc/zabbix/zabbix_server.conf &&
echo "DBSocket=/var/lib/mysql/mysql.sock" >> /etc/zabbix/zabbix_server.conf &&
echo "DBUser=zabbix" >> /etc/zabbix/zabbix_server.conf &&
echo "DBPassword=zabbixpassword" >> /etc/zabbix/zabbix_server.conf &&
echo "StartPollers=8" >> /etc/zabbix/zabbix_server.conf &&
echo "StartPollersUnreachable=2" >> /etc/zabbix/zabbix_server.conf &&
echo "StartTrappers=3" >> /etc/zabbix/zabbix_server.conf &&
echo "StartPingers=3" >> /etc/zabbix/zabbix_server.conf &&
echo "StartDiscoverers=1" >> /etc/zabbix/zabbix_server.conf &&
echo "StartHTTPPollers=2" >> /etc/zabbix/zabbix_server.conf &&
echo "HousekeepingFrequency=1" >> /etc/zabbix/zabbix_server.conf &&
echo "MaxHousekeeperDelete=1000" >> /etc/zabbix/zabbix_server.conf &&
echo "SenderFrequency=30" >> /etc/zabbix/zabbix_server.conf &&
echo "CacheSize=16M" >> /etc/zabbix/zabbix_server.conf &&
echo "CacheUpdateFrequency=60" >> /etc/zabbix/zabbix_server.conf &&
echo "StartDBSyncers=4" >> /etc/zabbix/zabbix_server.conf &&
echo "HistoryCacheSize=8M" >> /etc/zabbix/zabbix_server.conf &&
echo "TrendCacheSize=8M" >> /etc/zabbix/zabbix_server.conf &&
echo "HistoryTextCacheSize=16M" >> /etc/zabbix/zabbix_server.conf &&
echo "Timeout=4" >> /etc/zabbix/zabbix_server.conf &&
echo "TrapperTimeout=300" >> /etc/zabbix/zabbix_server.conf &&
echo "UnreachablePeriod=45" >> /etc/zabbix/zabbix_server.conf &&
echo "UnavailableDelay=60" >> /etc/zabbix/zabbix_server.conf &&
echo "UnreachableDelay=15" >> /etc/zabbix/zabbix_server.conf &&
echo "StartProxyPollers=0" >> /etc/zabbix/zabbix_server.conf &&
echo "ProxyConfigFrequency=1" >> /etc/zabbix/zabbix_server.conf &&
echo "ProxyDataFrequency=1" >> /etc/zabbix/zabbix_server.conf &&

echo "PidFile=/tmp/zabbix_agentd.pid" > /etc/zabbix/zabbix_agentd.conf &&echo "LogFileSize=1" >> /etc/zabbix/zabbix_agentd.conf &&
echo "DebugLevel=3" >> /etc/zabbix/zabbix_agentd.conf &&
echo "LogFile=/tmp/zabbix_agentd.log" >> /etc/zabbix/zabbix_agentd.conf &&
echo "EnableRemoteCommands=1" >> /etc/zabbix/zabbix_agentd.conf &&
echo "LogRemoteCommands=1" >> /etc/zabbix/zabbix_agentd.conf &&
echo "Server=xxx.xxx.xxx.xxx" >> /etc/zabbix/zabbix_agentd.conf &&
echo "Hostname=$HOSTNAME" >> /etc/zabbix/zabbix_agentd.conf &&
echo "StartAgents=3" >> /etc/zabbix/zabbix_agentd.conf &&
echo "Timeout=3" >> /etc/zabbix/zabbix_agentd.conf &&

echo "zabbix-agent 10050/tcp #Zabbix Agent" >> /etc/services &&
echo "zabbix-agent 10050/udp #Zabbix Agent" >> /etc/services &&
echo "zabbix-trapper 10051/tcp #Zabbix Trapper" >> /etc/services &&
echo "zabbix-trapper 10051/udp #Zabbix Trapper" >> /etc/services &&

mkdir /var/www/html/zabbix &&
cp -r /home/zabbix/zabbix-1.8.9/frontends/php/* /var/www/html/zabbix/

vim /etc/php.ini
max_execution_time = 600
max_input_time = 600
memory_limit = 256M
post_max_size = 32M
upload_max_filesize = 16M
date.timezone = "America/Sao_Paulo"
mbstring.func_overload = 2

vim /etc/httpd/conf/httpd.conf
NameVirtualHost *:80
<VirtualHost *:80>
DocumentRoot /var/www/html/zabbix
ServerName zabbix.domain.com
ErrorLog /var/log/httpd/zabbix.domain.com-error_log
CustomLog /var/log/httpd/zabbix.domain.com-access_log common
</VirtualHost>

Stopped all needed services and started conversion:
/usr/local/sbin/zabbix_server -n 1 -c /etc/zabbix/zabbix_server.conf

Then started the services and waited for data coming out.

chkconfig --levels 35 httpd on
chkconfig --levels 35 mysqld on

service mysqld restart
service httpd restart

/usr/local/sbin/zabbix_server
/usr/local/sbin/zabbix_agentd

Nothing yet.. the same problem of my another slave nodes.

I've tried restarting services, erasing the nodes in the frontend configuration and adding then again but nothing, they do connect well, but doesn't sync any data.

I will try to re-install this new Test Slave Node and convert again.

If this works I can do the same in my another nodes, and then restore the dump backup, but it will take so much time, get my nodes without any monitoring solutions and will slow down my other services that uses mysql too.

Can't we do something else to reestablish the sync from the servers, without damaging my nodes anymore?

-------------------------

2011 Dec 20 16:58
Re-installing didn't worked because the backups are obviously already converted to distributed monitoring.

So... if someone like me loses all the Master Node data, will need to do a reinstall on all the slave nodes?

How can I backup all history, events, triggers, confs, etc, etc without the "conversion configuration"? Maybe doing that and re-converting will solve the problem?

Thank you.

**adriano** · 26-12-2011, 16:46

no one even with the same problem?

I'm still searching and thinking but have no more ideas..

**adriano** · 27-12-2011, 01:44

more logs..

Tried reconfiguring Zabbix Master Node.

Have done a clean install in the master and tested syncing.. still not working..

Here is the Test Slave Node log:
Test Node NodeID=98
Master Node NodeID=5

Code:

  5234:20111226:213532.860 NODE 98: Sending configuration changes to master node 5 for node 98 datalen 3528
  5234:20111226:213535.642 NODE 98: Received configuration changes from master node 5 for node 98 datalen 6241
  5234:20111226:213536.541 [Z3005] query failed: [1062] Duplicate entry '0' for key 'user_history_1' [update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012768;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012769;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012770;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012771;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012772;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012773;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012774;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012775;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012776;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012777;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012778;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012779;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012780;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012781;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012782;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012783;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012784;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012785;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012786;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012788;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012789;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012790;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012791;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012792;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012793;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012795;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012796;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012797;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012798;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012800;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012801;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012802;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012803;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012804;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012805;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012806;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012807;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012808;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012809;
update triggers set expression='',description='',url='',status=0,value=0,priority=0,dep_level=0,comments='',error='',templateid=0,type=0 where triggerid=9809800000012810;
update triggers set expression='',description='',url='',status=0,priority=0,dep_level=0,comments='',templateid=0,type=0 where triggerid=9809800000012811;
update user_history set userid=0,title2='',url2='' where userhistoryid=9809800000000002;
]
  5234:20111226:213536.938 NODE 98: Sending history_sync of node 98 to node 5 datalen 9583
  5234:20111226:213537.236 NODE 98: Sending history_uint_sync of node 98 to node 5 datalen 20110
  5232:20111226:213537.524 Deleted 0 records from history and trends
  5234:20111226:213537.909 NODE 98: Sending history_str_sync of node 98 to node 5 datalen 3247
  5234:20111226:213538.260 NODE 98: Sending events of node 98 to node 5 datalen 31131
  5234:20111226:213538.602 NOT OK
  5234:20111226:213538.748 NODE 98: Sending auditlog of node 98 to node 5 datalen 19225
  5234:20111226:213539.073 NODE 98: Sending auditlog_details of node 98 to node 5 datalen 6916
  5234:20111226:213539.334 NODE 98: Sending history_sync of node 98 to node 5 datalen 491
  5234:20111226:213539.496 NODE 98: Sending history_uint_sync of node 98 to node 5 datalen 1150
  5234:20111226:213539.677 NODE 98: Sending history_str_sync of node 98 to node 5 datalen 126
  5234:20111226:213539.970 NODE 98: Sending events of node 98 to node 5 datalen 31131
  5234:20111226:213540.309 NOT OK

**adriano** · 27-12-2011, 03:10

Done!?!?!

I guess this one fixed:
http://www.zabbix.com/wiki/doc/troubleshooting/index

Code:

* 1. Stop Master node
* 2. Execute on NODEx (not MASTER):
  *delete from node_cksum;
  *delete from node_configlog;
* 3. Start Master node

I've restarted both sides - master and slave - after the workaround, everything looks fine now, great Zabbix is back at monitoring, but I don't know if this is the only thing needed to work, because I've tested so much things at the same time.. heheh

I will better test this tomorrow, reproduce the error and make everything again.

I'll post any errors, if found.

I guess this "General Troubleshooting" need some more info, because we have the solution right at hand, but there is nothing about the errors that it solves.

Sorry about my English.

Kind regards.

**adriano** · 29-12-2011, 18:31

everything ok

Hi!

Sorry about the delay, was busy and didn't had so much time to come here again.

Everything is working fine now, thanks for the tips in the wiki.

Kind regards.

Ad Widget

Node sync problems

Node sync problems

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment