Ad Widget

Collapse

Zabbix child does not process config data

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • cookiemonster
    Junior Member
    • Jul 2014
    • 5

    #1

    Zabbix child does not process config data

    Hello and thank you for your input on this thread.

    I have recently added a third child in my Zabbix DM configuration.
    During the first few hours after set up all was nice and dandy. After the weekend I have discovered that the child was not syncing properly. It is receiving the configuration updates from the master but I cannot tell from the logs that it does anything with them.

    Here are the logs on the master:
    --
    6724:20140728:144818.402 NODE 1: sending configuration changes to slave node 12 for node 12 datalen 13229230
    --
    6724:20140728:144921.081 NODE 1: sending configuration changes to slave node 12 for node 12 datalen 13228736
    --
    6721:20140728:150056.359 NODE 1: sending configuration changes to slave node 12 for node 12 datalen 13228736
    6721:20140728:150556.360 NODE 1: cannot receive answer from Node [12]: ZBX_TCP_READ() failed: [4] Interrupted system call

    And the slave:
    2582:20140728:144912.574 NODE 12: sending configuration changes to master node 1 for node 12 datalen 1543
    2582:20140728:144934.711 NODE 12: Received configuration changes from master node 1 for node 12 datalen 13228736
    --
    2582:20140728:150046.820 NODE 12: sending configuration changes to master node 1 for node 12 datalen 4873
    2582:20140728:150117.111 NODE 12: Received configuration changes from master node 1 for node 12 datalen 13228736

    The "ZBX_TCP_READ()" failure seems to be caused by the child itself that sends a RST back to the master after 5 minutes. Can't tell why.

    I see the data sent by the master in the child's log with debug level 4, but it does not log anything about doing something with it. Hence every 10-12 minutes, the master keeps sending the same data.

    Also the same child seems to be having issues sending the history and trends to the master (currently the master is missing the last 36 hours of history from this child).

    All servers are running Zabbix 2.2.3 (master on Red Hat and the rest of the children on Solaris 10).

    Does anybody have any idea why this child ignores the configuration updates?
  • cookiemonster
    Junior Member
    • Jul 2014
    • 5

    #2
    Updates

    So, I found that the RST is being sent by the master after it times out because the child did not return in 5 minutes (apparently value controlled by trappertimeout that cannot go over 300 seconds).
    I cannot speed up the database on the child (MySQL 5.0 32 bit).

    The issue seems to be uncovered here (since Zabbix 1.6)


    Adding the index does not help.

    Is there another option beside recompiling from the source with a different value hardcoded for the timeout?

    Thank you

    Comment

    • cookiemonster
      Junior Member
      • Jul 2014
      • 5

      #3
      Possible solution

      Ok, so after much tinkering with the MySQL set up, I could not improve the speed of it or make Zabbix process the configuration update faster.
      I ended up emptying the node_cksum table on the child node and this seem to have re-established the sync between the nodes.
      I will have to verify that all hosts are linked properly with the templates on the child node and redo the links where there is a mismatch between the child and the master.

      Comment

      Working...