Ad Widget

Collapse

FYI Upgrade woes slow mysql and how I recovered

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • swisspace
    Junior Member
    Zabbix Certified Specialist
    • May 2011
    • 3

    #1

    FYI Upgrade woes slow mysql and how I recovered

    Just posting this in case anyone else is as foolish as me :-)

    Upgrade of zabbix from 1.8 to 2.0

    read installation notes etc and make mysql all databases dump which took about 6 hours and resulted in a 10Gb file.

    I should add that we are a reasonably small company with no test environment so I thought I would do the upgrade straight on the productive environment.

    start patch.sql whilst doing make install etc and copying frontends (after tarring them up). Fine finished binary and frontend wait for patch.sql to finish.

    1 day later hmm this is taking a bit long do some searching aha need to finetune mysql etc but I will wait.

    2 days later still running hmmm more searching and I See mention of 11 or 18 days for this script to run. 4 days later Hmm not sure if this will ever finish so I kill it.

    Dump only took a few hours so I will restore that and do the two stage upgrade
    detailed here - http://www.zabbix.com/forum/showthre...ferrerid=50478

    Hmm 4 days later but this time armed with mysql workbench and having bothered to read more about mysql I did at least see that rows were being added, this I thought can't be right and no zabbix alerting now for over a week i decided to edit the mysql dump file (10Gb) and remove the history data which editing a 10Gb file is a problem in itself, having found ultredit being the only editor capable on my mac to open it and then extremely slow to edit I had to find another way.

    Command line grep I used the following

    $ grep -v "INSERT INTO \`alerts\` VALUES" zabbNOhist.sql > zabb2.sql
    which i repeated for all the tables detailed in the thread I detailed above

    having got the file to a more manageable 3.4mb I then restored this file and reran the patch.sql script which stopped a few times but deleting the problem lines solved that.

    Restarted the sever and completed the install screen to create new config file and its all back up and running minus the history.

    What I plan to do now is create a test environment and see if I can speed up the mysql db restore patching etc.. or move to another db BUT I will abandon adding the history bak into the productive system - although nice I can live without it but I will want it carried over when I do the next upgrade, hence the research now.

    For anyone thinking what a plonker, there may be some truth in that but in mitigation I didn't consider the db large at 10Gigs and as it was so quick to dump I never envisaged that it would take days to restore, upgrade etc. especially as mysql is supposed to be a good db, However I should have taken a copy of the virtual machine and ran a test there which had it been a company critical app I would have done.
Working...