Zabbix 1.6.1 - Not Syncing Between Nodes

[abhi238]

Hi,

I am trying to configure Master/Child Nodes setup but the syncing of nodes does not work.

I noticed a thread with the same issue for 1.4.1 and they provided a fix but same fix cannot be tried for the 1.6.1 "nodewatcher/nodesender.c" as code seems to have changed. The same problem is noticed with zabbix 1.6.1 and not fixed.

From the Master node Web Interface, if I select child node then I get error "Unable to select configuration" and vice-versa.
From both the nodes with verbose=4, I see only starting sync messages but not doing actual sync:

Below is the snippet of the zabbix_server.log file:
>>>>>>>>>>>>>>>>>
1459:20081217:170347 Starting sync with nodes
1459:20081217:170357 Starting sync with nodes
1459:20081217:170407 Starting sync with nodes
1459:20081217:170417 Starting sync with nodes
1459:20081217:170427 Starting sync with nodes
1459:20081217:170437 Starting sync with nodes
1459:20081217:170447 Starting sync with nodes
1459:20081217:170457 Starting sync with nodes
1459:20081217:170507 Starting sync with nodes
>>>>>>>>>>>>>>>>>


Kindly advice,

Thanks

======================
Zabbix 1.6.1:
Monitored Hosts/Items : 1700/20000
Monitored Host/Items (Proxy) : 1045/13500
======================

[abhi238]

ok. I resolved this.


The issue was with the mismatch of the masterid's of both master/child on the mysql DB on both master/child.

[wakko]

Can you give a more detailed description of the solution? I'm having the same issue.

[abhi238]

Hi,

You need to see that the masterid in the nodes table on both master/child is same.
I.E.: Run the below query on both the master/child nodes DB and verify if they are the same. They should be the same or else you will face the sync issue.

mysql> select nodeid,name,masterid from nodes;
+--------+-------------+----------+
| nodeid | name | masterid |
+--------+-------------+----------+
| 1 | Master node | 0 |
| 2 | Child Node | 1 |
+--------+-------------+----------+
2 rows in set (0.00 sec)



Hope this helps,

Regards,
Abhijit

[wakko]

Thank you for the additional details.

It looks like I have the same problem, but updating the nodes table to align the values doesn't seem to have helped. Were there any other steps you took?

[abhi238]

Ok. This is how I setup the Master/Child:

Master IP/Port: <IP>/10051
child IP/Port: <IP>/10051

On Master:
1. Set NodeID=1 in the /etc/zabbix/zabbix_server.conf.
2. Exec: zabbix_server -n 1 -c /etc/zabbix/zabbix_server.conf
3. From Master Web Interface:
a) Added/Updated the Master Node details from Administration->Nodes Tab
b) Added/Updated the Child Node details

On Child:
1. Set NodeID=2 in the /etc/zabbix/zabbix_server.conf.
2. Exec: zabbix_server -n 2 -c /etc/zabbix/zabbix_server.conf
3. From Master Web Interface:
a) Updated the Master Node details from Administration->Nodes Tab
b) Added/Updated the Child Node details


Verified the masterid column values are identical in the NODES tables for both master/child's database.

Start Zabbix server daemons on both and verify /tmp/zabbix_server.log

You should see something similar in the Master logs if sync works:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>

25489:20081218:211841 NODE 1: Received data from slave node 2 for node 2 datalen 8
25485:20081218:211849 NODE 1: Received history from node 2 for node 2 datalen 126
25487:20081218:211849 NODE 1: Received history_uint from node 2 for node 2 datalen 342
25487:20081218:211856 NODE 1: Received history_uint from node 2 for node 2 datalen 209
25493:20081218:211857 NODE 1: Received history_str from node 2 for node 2 datalen 102
25493:20081218:211906 NODE 1: Received history from node 2 for node 2 datalen 126
25491:20081218:211907 NODE 1: Received history_uint from node 2 for node 2 datalen 250
25491:20081218:211917 NODE 1: Received history from node 2 for node 2 datalen 58
25489:20081218:211917 NODE 1: Received history_uint from node 2 for node 2 datalen 255
25485:20081218:211918 NODE 1: Received history_str from node 2 for node 2 datalen 94
25485:20081218:211926 NODE 1: Received history from node 2 for node 2 datalen 175
25487:20081218:211927 NODE 1: Received history_uint from node 2 for node 2 datalen 154
25493:20081218:211927 NODE 1: Received history_str from node 2 for node 2 datalen 156
25489:20081218:212050 NODE 1: Received data from slave node 2 for node 2 datalen 8
25491:20081218:212258 NODE 1: Received data from slave node 2 for node 2 datalen 8
25493:20081218:212505 NODE 1: Received data from slave node 2 for node 2 datalen 8
25487:20081218:212712 NODE 1: Received data from slave node 2 for node 2 datalen 8
>>>>>>>>>>>>>>>>>>>>>>>>>>>>

On the Child Node /tmp/zabbix_server.log you will notice similar messages as seen below:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
1530:20081223:101054 NODE 2: Sending configuration changes to master node 1 for node 2 datalen 9
1530:20081223:101059 NODE 2: Received data from master node 1 for node 2 datalen 151
1530:20081223:101102 NODE 2: Sending history_sync of node 2 to node 1 datalen 413
1530:20081223:101103 NODE 2: Sending history_uint_sync of node 2 to node 1 datalen 1271
1530:20081223:101104/ NODE 2: Sending history_str_sync of node 2 to node 1 datalen 296
1530:20081223:101107 NODE 2: Sending trends of node 2 to node 1 datalen 89
1530:20081223:101109 NODE 2: Sending trends_uint of node 2 to node 1 datalen 64
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

If this does not help, let me know more details of your setup and output of below queries on both master/child:

1. select nodeid,name,masterid from nodes;

2. select userid from users;


Happy Holidays!!

[wakko]

Weird. I'd swear those are the steps I followed earlier today.

However, I redid them again and now it works.

Now to sort out the ACLs on the switches to allow the traffic through.


Thanks for your help!