Ad Widget

Collapse

Oracle and zabbix

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Italo Fernando Lopes
    Member
    • Feb 2011
    • 58

    #1

    Oracle and zabbix

    Can anyone help me?
    I am using zabbix server with 4813 hosts
    4 proxies
    My question is:
    What would be the correct configuration for the housekeeper in both the proxy when the server
    Last edited by Italo Fernando Lopes; 22-02-2011, 20:48.
  • Italo Fernando Lopes
    Member
    • Feb 2011
    • 58

    #2
    Originally posted by Italo Fernando Lopes
    Can anyone help me?
    When running the housekeeper has Database is Down message.
    I was informed by the team of the company that Oracle displays error ORA-07041 due to the size of the housekeeper (delete million is being executed)
    Already configured Maxhousekeeper: 500 lowered to 50 for 5 and 2 to solve anything.
    My server is 4830 with equipment items 34 000 and 182 VPS.
    Any suggestions? They told me I have to change the server.c and this true?
    Alexei help me?

    Comment

    • untergeek
      Senior Member
      Zabbix Certified Specialist
      • Jun 2009
      • 512

      #3
      Housekeeper is great for small installs. It sounds like yours isn't.

      Your options include:

      Partitioned tables. Drop them at specified times, e.g. drop the history tables for the month of January once you've reached the month of March. Do the same for each month's worth of trend data, but at the 13 month mark (keep a year of trend). You could tailor that to meet your needs.

      Straight out "delete history older than x days" rather than the per item querying that the housekeeper does. I have some simple scripts I use to do this for us right now while we transition to the partitioned table solution.

      Table drops are much easier on Oracle than queries and deletes.

      Comment

      • Italo Fernando Lopes
        Member
        • Feb 2011
        • 58

        #4
        Originally posted by untergeek
        Housekeeper is great for small installs. It sounds like yours isn't.

        Your options include:

        Partitioned tables. Drop them at specified times, e.g. drop the history tables for the month of January once you've reached the month of March. Do the same for each month's worth of trend data, but at the 13 month mark (keep a year of trend). You could tailor that to meet your needs.

        Straight out "delete history older than x days" rather than the per item querying that the housekeeper does. I have some simple scripts I use to do this for us right now while we transition to the partitioned table solution.

        Table drops are much easier on Oracle than queries and deletes.
        Thanks for the reply was just what I imagined.
        You send me the example of a scrip?
        Previously I had ORACLE given me the suggestion to change the limit on DB.C.
        There was very nice.

        Comment

        • untergeek
          Senior Member
          Zabbix Certified Specialist
          • Jun 2009
          • 512

          #5
          This is what I use. You will probably want to tailor this to suit your needs.

          Code:
          #!/bin/bash 
          
          SID=ORACLE_SID
          USERNAME="dbuser"
          PASSWORD="dbpassword"
          
          ZABBIX_SERVER=zabbix-server.example.com
          ZABBIX_HOST="Zabbix Server"
          
          . /opt/oracle/product/11.2/oracle.env
          
          export ORACLE_SID="${SID}"
          
          
          NOW=$(date +%s)
          JobSTART_TIME=$(date +%s.%N)
          LAST_WEEK=$((NOW-3600*24*7))
          LAST_MONTH=$((NOW-3600*24*30))
          LAST_YEAR=$((NOW-3600*24*365))
          TOTAL_ROWS=0
          
          query_get ()
          #  Pass query body
          #  run query and parse
          #  spit out results
          {   
             local QUERY
             local START_TIME=$(date +%s.%N)
             case $1 in 
                alerts            ) QUERY="delete from alerts where clock<${LAST_YEAR}";;
                history           ) QUERY="delete from history where clock<${LAST_MONTH}";;
                history_uint      ) QUERY="delete from history_uint where clock<${LAST_MONTH}";;
                history_str       ) QUERY="delete from history_str where clock<${LAST_WEEK}";;
                history_text      ) QUERY="delete from history_text where clock<${LAST_WEEK}";;
                history_log       ) QUERY="delete from history_log where clock<${LAST_WEEK}";;
                trends            ) QUERY="delete from trends where clock<${LAST_YEAR}";;
                trends_uint       ) QUERY="delete from trends_uint where clock<${LAST_YEAR}";;
                acknowledges      ) QUERY="delete from acknowledges where eventid=(select eventid from events where clock<${LAST_YEAR})";;
                events            ) QUERY="delete from events where eventid=(select eventid from events where clock<${LAST_YEAR})";;
                *                 ) echo "Fail.";exit 1;;
             esac
             local SPOOL=/var/tmp/query_get.$RANDOM
             local getResult=$(sqlplus -S ${USERNAME}/${PASSWORD}@${ORACLE_SID} << EOF
             set pagesize 100
             set linesize 100
             set heading off
             set tab off
             spool $SPOOL
             ${QUERY};
             spool off
             commit;
             quit
             EOF)
          
             local END_TIME=$(date +%s.%N)
             local ELAPSED=$(printf "%.3F"  $(echo "$END_TIME - $START_TIME"|bc))
          
             local ROWS=$(grep rows $SPOOL | awk '{print $1}')
             TOTAL_ROWS=$((TOTAL_ROWS + ROWS))
          
             echo "/usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s \"${ZABBIX_HOST}\" -k hk.rows_deleted[${1}] -o ${ROWS}"
             /usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s "${ZABBIX_HOST}" -k hk.rows_deleted[${1}] -o ${ROWS}
             echo "/usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s \"${ZABBIX_HOST}\" -k hk.time_elapsed[${1}] -o ${ELAPSED}"
             /usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s "${ZABBIX_HOST}" -k hk.time_elapsed[${1}] -o ${ELAPSED}
             rm $SPOOL
             return
          }
          
          #echo
          #echo -n "Time now: "
          #echo $NOW| perl -e 'print localtime(<>) . "\n";'
          #echo -n "Deleting history older than: "
          #echo $LAST_MONTH| perl -e 'print localtime(<>) . "\n";'
          #echo -n "Deleting trends older than: "
          #echo $LAST_YEAR| perl -e 'print localtime(<>) . "\n";'
          #echo
          
          
          for TABLE in acknowledges alerts events history history_log history_str history_text history_uint trends trends_uint; do
             query_get $TABLE
          done
          
          TOTAL_ELAPSED=$(printf "%.3F"  $(echo "$(date +%s.%N) - $JobSTART_TIME"|bc))
          
          echo "/usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s \"${ZABBIX_HOST}\" -k hk.rows_deleted[TOTAL] -o ${TOTAL_ROWS}"
          /usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s "${ZABBIX_HOST}" -k hk.rows_deleted[TOTAL] -o ${TOTAL_ROWS}
          echo "/usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s \"${ZABBIX_HOST}\" -k hk.time_elapsed[TOTAL] -o ${TOTAL_ELAPSED}"
          /usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s "${ZABBIX_HOST}" -k hk.time_elapsed[TOTAL] -o ${TOTAL_ELAPSED}
          As you can see, there are zabbix_sender items so I can trend how many rows were deleted from each table and how long it took for the whole process.

          This mimics the regular housekeeping processes to the letter, except that instead of figuring out how long each item is to be kept based on its history and trend value (in the individual item defs) we keep a fixed 30 days for history and history_uint, 7 days for char/text/log data, and 365 days for trend data. This dramatically reduced the number of individual queries being made as we have over 25,000 individual items

          You'll need to make items in the host called "Zabbix Server" (or whatever other host you'd prefer to use) with keys of hk.rows_deleted[*] and hk.time_elapsed[*] where the * can be TOTAL, alerts, history, history_uint, history_str, history_text, history_log, trends, trends_uint, acknowledges or events. That way you trend how long it takes to delete from each table and how many rows were deleted from each.

          Again, this was written so we had an interim solution before we migrate to using partitioned tables. Your mileage may vary.

          Comment

          • Italo Fernando Lopes
            Member
            • Feb 2011
            • 58

            #6
            Originally posted by untergeek
            This is what I use. You will probably want to tailor this to suit your needs.

            Code:
            #!/bin/bash 
            
            SID=ORACLE_SID
            USERNAME="dbuser"
            PASSWORD="dbpassword"
            
            ZABBIX_SERVER=zabbix-server.example.com
            ZABBIX_HOST="Zabbix Server"
            
            . /opt/oracle/product/11.2/oracle.env
            
            export ORACLE_SID="${SID}"
            
            
            NOW=$(date +%s)
            JobSTART_TIME=$(date +%s.%N)
            LAST_WEEK=$((NOW-3600*24*7))
            LAST_MONTH=$((NOW-3600*24*30))
            LAST_YEAR=$((NOW-3600*24*365))
            TOTAL_ROWS=0
            
            query_get ()
            #  Pass query body
            #  run query and parse
            #  spit out results
            {   
               local QUERY
               local START_TIME=$(date +%s.%N)
               case $1 in 
                  alerts            ) QUERY="delete from alerts where clock<${LAST_YEAR}";;
                  history           ) QUERY="delete from history where clock<${LAST_MONTH}";;
                  history_uint      ) QUERY="delete from history_uint where clock<${LAST_MONTH}";;
                  history_str       ) QUERY="delete from history_str where clock<${LAST_WEEK}";;
                  history_text      ) QUERY="delete from history_text where clock<${LAST_WEEK}";;
                  history_log       ) QUERY="delete from history_log where clock<${LAST_WEEK}";;
                  trends            ) QUERY="delete from trends where clock<${LAST_YEAR}";;
                  trends_uint       ) QUERY="delete from trends_uint where clock<${LAST_YEAR}";;
                  acknowledges      ) QUERY="delete from acknowledges where eventid=(select eventid from events where clock<${LAST_YEAR})";;
                  events            ) QUERY="delete from events where eventid=(select eventid from events where clock<${LAST_YEAR})";;
                  *                 ) echo "Fail.";exit 1;;
               esac
               local SPOOL=/var/tmp/query_get.$RANDOM
               local getResult=$(sqlplus -S ${USERNAME}/${PASSWORD}@${ORACLE_SID} << EOF
               set pagesize 100
               set linesize 100
               set heading off
               set tab off
               spool $SPOOL
               ${QUERY};
               spool off
               commit;
               quit
               EOF)
            
               local END_TIME=$(date +%s.%N)
               local ELAPSED=$(printf "%.3F"  $(echo "$END_TIME - $START_TIME"|bc))
            
               local ROWS=$(grep rows $SPOOL | awk '{print $1}')
               TOTAL_ROWS=$((TOTAL_ROWS + ROWS))
            
               echo "/usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s \"${ZABBIX_HOST}\" -k hk.rows_deleted[${1}] -o ${ROWS}"
               /usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s "${ZABBIX_HOST}" -k hk.rows_deleted[${1}] -o ${ROWS}
               echo "/usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s \"${ZABBIX_HOST}\" -k hk.time_elapsed[${1}] -o ${ELAPSED}"
               /usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s "${ZABBIX_HOST}" -k hk.time_elapsed[${1}] -o ${ELAPSED}
               rm $SPOOL
               return
            }
            
            #echo
            #echo -n "Time now: "
            #echo $NOW| perl -e 'print localtime(<>) . "\n";'
            #echo -n "Deleting history older than: "
            #echo $LAST_MONTH| perl -e 'print localtime(<>) . "\n";'
            #echo -n "Deleting trends older than: "
            #echo $LAST_YEAR| perl -e 'print localtime(<>) . "\n";'
            #echo
            
            
            for TABLE in acknowledges alerts events history history_log history_str history_text history_uint trends trends_uint; do
               query_get $TABLE
            done
            
            TOTAL_ELAPSED=$(printf "%.3F"  $(echo "$(date +%s.%N) - $JobSTART_TIME"|bc))
            
            echo "/usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s \"${ZABBIX_HOST}\" -k hk.rows_deleted[TOTAL] -o ${TOTAL_ROWS}"
            /usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s "${ZABBIX_HOST}" -k hk.rows_deleted[TOTAL] -o ${TOTAL_ROWS}
            echo "/usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s \"${ZABBIX_HOST}\" -k hk.time_elapsed[TOTAL] -o ${TOTAL_ELAPSED}"
            /usr/local/bin/zabbix_sender -z ${ZABBIX_SERVER} -s "${ZABBIX_HOST}" -k hk.time_elapsed[TOTAL] -o ${TOTAL_ELAPSED}
            As you can see, there are zabbix_sender items so I can trend how many rows were deleted from each table and how long it took for the whole process.

            This mimics the regular housekeeping processes to the letter, except that instead of figuring out how long each item is to be kept based on its history and trend value (in the individual item defs) we keep a fixed 30 days for history and history_uint, 7 days for char/text/log data, and 365 days for trend data. This dramatically reduced the number of individual queries being made as we have over 25,000 individual items

            You'll need to make items in the host called "Zabbix Server" (or whatever other host you'd prefer to use) with keys of hk.rows_deleted[*] and hk.time_elapsed[*] where the * can be TOTAL, alerts, history, history_uint, history_str, history_text, history_log, trends, trends_uint, acknowledges or events. That way you trend how long it takes to delete from each table and how many rows were deleted from each.

            Again, this was written so we had an interim solution before we migrate to using partitioned tables. Your mileage may vary.
            Thanks for the script
            Last question I run it directly from the database server?

            Comment

            • untergeek
              Senior Member
              Zabbix Certified Specialist
              • Jun 2009
              • 512

              #7
              Or anywhere with TCP access to the database.

              Comment

              Working...