Ad Widget

Collapse

Zabbix upgrade failure from 3.2 to 3.4

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    Zabbix upgrade failure from 3.2 to 3.4

    Server info:
    Code:
    Operating system:
    Ubuntu 16.04.4 LTS
    
    Current Zabbix server version:
    zabbix_server (Zabbix) 3.2.11
    Revision 76339 27 December 2017, compilation time: Dec 27 2017 15:21:29
    
    Zabbix NVPS:
    1245
    
    Database:
    MySQL 5.7
    Problem 1
    We tried to upgrade our current zabbix version to 3.4 and initially got below database related errors
    Code:
    1867:20181031:060213.293 [Z3005] query failed: [1091] Can't DROP 'c_dservices_2'; check that column/key exists [alter table dservices drop foreign key c_dservices_2]
    For this we ran below query and it got fixed
    Code:
    ALTER TABLE dservices ADD CONSTRAINT c_dservices_2 FOREIGN KEY (dcheckid) REFERENCES dchecks (dcheckid) ON DELETE CASCADE;
    Problem 2
    During the upgrade we got another error like below
    Code:
    22286:20181031:122343.855 [Z3005] query failed: [1452] Cannot add or update a child row: a foreign key constraint fails (`zabbix`.`alerts`, CONSTRAINT `c_alerts_5` FOREIGN KEY (`p_eventid`) REFERENCES `events` (`eventid`) ON DELETE CASCADE) [update alerts set p_eventid=1042260 where eventid=1042312;
    we tried to delete this record from alerts table and it got cleared, but same error appeared with a different event id and so on. Since we have large number of events this was a difficult and time consuming task to check each time upgrade fails and we delete these records one by one.

    We tried two methods to get around this
    Option 1
    We tried to turn off foreign key check.

    Option 2
    We removed trigger data from events and alerts table table
    Code:
    delete from alerts;
    delete from events where source=0;
    With both options at each attempts (we're testing this upgrade with a snapshot of production) , database upgrade completed to 100% and right after that we got below same error from the logs
    Code:
    12460:20181102:115613.150 completed 66% of database upgrade
     12460:20181102:115613.235 completed 83% of database upgrade
     12460:20181102:115613.236 completed 100% of database upgrade
     12460:20181102:115613.236 database upgrade fully completed
     12460:20181102:115626.508 slow query: 13.218710 sec, "select i.itemid,i.hostid,i.status,i.type,i.value_type,i.key_,i.snmp_community,i.snmp_oid,i.port,i.snmpv3_securityname,i.snmpv3_securitylevel,i.snmpv3_authpassphrase,i.snmpv3_privpassphrase,i.ipmi_sensor,i.delay,i.trapper_hosts,i.logtimefmt,i.params,i.state,i.authtype,i.username,i.password,i.publickey,i.privatekey,i.flags,i.interfaceid,i.snmpv3_authprotocol,i.snmpv3_privprotocol,i.snmpv3_contextname,i.lastlogsize,i.mtime,i.history,i.trends,i.inventory_link,i.valuemapid,i.units,i.error,i.jmx_endpoint,i.master_itemid from items i,hosts h where i.hostid=h.hostid and h.status in (0,1) and i.flags<>2"
    zabbix_server [12460]: ERROR [file:dbconfig.c,line:1999] Something impossible has just happened.
    zabbix_server [12460]: ERROR [file:dbconfig.c,line:1999] Something impossible has just happened.
    zabbix_server [12460]: ERROR [file:dbconfig.c,line:1999] Something impossible has just happened.
    zabbix_server [12460]: ERROR [file:dbconfig.c,line:1999] Something impossible has just happened.
    zabbix_server [12460]: ERROR [file:dbconfig.c,line:1999] Something impossible has just happened.
    zabbix_server [12460]: ERROR [file:dbconfig.c,line:1999] Something impossible has just happened.
    zabbix_server [12460]: ERROR [file:dbconfig.c,line:1999] Something impossible has just happened.
    zabbix_server [12460]: ERROR [file:dbconfig.c,line:1999] Something impossible has just happened.
    Then server process started and stopped with below error
    Code:
     25387:20181031:153429.034 syncing history data...
     25387:20181031:153429.034 syncing history data done
     25387:20181031:153429.034 syncing trend data...
     25387:20181031:153429.034 syncing trend data done
     25387:20181031:153429.036 Zabbix Server stopped. Zabbix 3.4.14 (revision 84877).
    zabbix_server [6976]: [file:'log.c',line:246] lock failed: [22] Invalid argument
    I'd be great if anyone could help us to get it fixed, or has anyone already experienced the same problem before and fixed it?

    #2
    could anyone shed some light?

    Comment


      #3
      Hi,
      https://support.zabbix.com/browse/ZBX-12657 and
      https://support.zabbix.com/browse/ZBX-14445
      These two could have traces. Please have a look there and see if that is your case.

      Comment

      Announcement

      Collapse
      No announcement yet.
      Working...
      X