Ad Widget

Collapse

Zabbix 1.8.8 crash

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • LPby
    Junior Member
    • Aug 2008
    • 21
    #1

    Zabbix 1.8.8 crash

    Установлен Zabbix 1.8.8 на CentOS5 i686 из исходников. Недавно изменил настройки MySQL, с бекапом и восстановлением после изменений БД. Изменения такие:
    Code:
    innodb_file_per_table
table_cache=128
query_cache_type=1
query_cache_limit=1M
query_cache_size=32M
tmpdir=/dev/shm
    После перезапуска пару часов все работало нормально. Затем, каждые 10 мин (xx:01, xx:11, xx:21, xx:31 и т.д.) сервер крэшится. Вот лог:
    Code:
    ......
26859:20111013:163108.292 In DBget_history()
 26859:20111013:163108.292 query [txnlev:1] [select avg(value) from history where itemid=22265 and clock>1318512546 and clock<=1318512666]
 26859:20111013:163108.293 Got signal [signal:11(SIGSEGV),reason:1,refaddr:(nil)]. Crashing ...
 26859:20111013:163108.293 ====== Fatal information: ======
 26859:20111013:163108.293 Program counter: 0x373453
 26859:20111013:163108.293 === Registers: ===
 26859:20111013:163108.293 gs      =               33 =                   51 =                   51
 26859:20111013:163108.293 fs      =                0 =                    0 =                    0
 26859:20111013:163108.293 es      =         c040007b =           3225419899 =          -1069547397
 26859:20111013:163108.293 ds      =         c040007b =           3225419899 =          -1069547397
 26859:20111013:163108.293 edi     =                0 =                    0 =                    0
 26859:20111013:163108.293 esi     =                0 =                    0 =                    0
 26859:20111013:163108.293 ebp     =         bfc694b8 =           3217462456 =          -1077504840
 26859:20111013:163108.293 esp     =         bfc6949c =           3217462428 =          -1077504868
 26859:20111013:163108.293 ebx     =           457ff4 =              4554740 =              4554740
 26859:20111013:163108.293 edx     =          92bea20 =            153872928 =            153872928
 26859:20111013:163108.293 ecx     =                0 =                    0 =                    0
 26859:20111013:163108.293 eax     =                0 =                    0 =                    0
 26859:20111013:163108.293 trapno  =                e =                   14 =                   14
 26859:20111013:163108.293 err     =                4 =                    4 =                    4
 26859:20111013:163108.293 eip     =           373453 =              3617875 =              3617875
 26859:20111013:163108.293 cs      =               73 =                  115 =                  115
 26859:20111013:163108.293 efl     =           210246 =              2163270 =              2163270
 26859:20111013:163108.293 uesp    =         bfc6949c =           3217462428 =          -1077504868
 26859:20111013:163108.293 ss      =               7b =                  123 =                  123
 26859:20111013:163108.293 === Stack frame: ===
 26859:20111013:163108.293 +0x40(%ebp) = ebp + 64 = 00000000 =          0 =           0
 26859:20111013:163108.293 +0x3c(%ebp) = ebp + 60 = 0000081f =       2079 =        2079
 26859:20111013:163108.294 +0x38(%ebp) = ebp + 56 = 080c3af9 =  135019257 =   135019257
 26859:20111013:163108.294 +0x34(%ebp) = ebp + 52 = 080a6192 =  134898066 =   134898066
 26859:20111013:163108.294 +0x30(%ebp) = ebp + 48 = bfc69758 = 3217463128 = -1077504168
 26859:20111013:163108.294 +0x2c(%ebp) = ebp + 44 = 00000000 =          0 =           0
 26859:20111013:163108.294 +0x28(%ebp) = ebp + 40 = 00000001 =          1 =           1
 26859:20111013:163108.294 +0x24(%ebp) = ebp + 36 = 0000005c =         92 =          92
 26859:20111013:163108.294 +0x20(%ebp) = ebp + 32 = 7156e100 = 1901519104 =  1901519104
 26859:20111013:163108.294 +0x1c(%ebp) = ebp + 28 = 0033c616 =    3393046 =     3393046
 26859:20111013:163108.294 +0x18(%ebp) = ebp + 24 = 00000000 =          0 =           0
 26859:20111013:163108.294 +0x14(%ebp) = ebp + 20 = bfc694fc = 3217462524 = -1077504772
 26859:20111013:163108.294 +0x10(%ebp) = ebp + 16 = 080c3bb2 =  135019442 =   135019442
 26859:20111013:163108.294 +0x0c(%ebp) = ebp + 12 = bfc694f4 = 3217462516 = -1077504780
 26859:20111013:163108.294 +0x08(%ebp) = ebp +  8 = 00000000 =          0 =           0 <--- call arguments
 26859:20111013:163108.294 +0x04(%ebp) = ebp +  4 = 0808b79a                            <--- return address
 26859:20111013:163108.294      (%ebp) = ebp      = bfc694e8                            <--- saved ebp value
 26859:20111013:163108.294 -0x04(%ebp) = ebp -  4 = 092bea20 =  153872928 =   153872928 <--- local variables
 26859:20111013:163108.294 -0x08(%ebp) = ebp -  8 = 00000000 =          0 =           0
 26859:20111013:163108.294 -0x0c(%ebp) = ebp - 12 = 00000009 =          9 =           9
 26859:20111013:163108.294 -0x10(%ebp) = ebp - 16 = 00000000 =          0 =           0
 26859:20111013:163108.294 -0x14(%ebp) = ebp - 20 = bfc694fc = 3217462524 = -1077504772
 26859:20111013:163108.294 -0x18(%ebp) = ebp - 24 = 00000000 =          0 =           0
 26859:20111013:163108.294 -0x1c(%ebp) = ebp - 28 = 00373195 =    3617173 =     3617173
 26859:20111013:163108.294 -0x20(%ebp) = ebp - 32 = bfc694b8 = 3217462456 = -1077504840
 26859:20111013:163108.294 -0x24(%ebp) = ebp - 36 = 0080cd00 =    8441088 =     8441088
 26859:20111013:163108.294 -0x28(%ebp) = ebp - 40 = 0000adb8 =      44472 =       44472
 26859:20111013:163108.294 -0x2c(%ebp) = ebp - 44 = 00000000 =          0 =           0
 26859:20111013:163108.294 -0x30(%ebp) = ebp - 48 = 00000000 =          0 =           0
 26859:20111013:163108.294 -0x34(%ebp) = ebp - 52 = 00000000 =          0 =           0
 26859:20111013:163108.295 -0x38(%ebp) = ebp - 56 = 00000000 =          0 =           0
 26859:20111013:163108.295 -0x3c(%ebp) = ebp - 60 = 00000000 =          0 =           0
 26859:20111013:163108.295 -0x40(%ebp) = ebp - 64 = 00000000 =          0 =           0
 26859:20111013:163108.295 === Backtrace: ===
 26858:20111013:163108.295 Syncing ...
 26858:20111013:163108.295 In DCsync_history() history_first:3505 history_num:0
 26858:20111013:163108.295 history syncer #3 spent 0.000050 seconds while processing 0 items
 26858:20111013:163108.295 sleeping for 5 seconds
 26859:20111013:163108.296 16: /usr/local/sbin/zabbix_server(print_fatal_info+0x311) [0x8086ed1]
 26859:20111013:163108.296 15: /usr/local/sbin/zabbix_server [0x8086434]
 26859:20111013:163108.296 14: [0x67e440]
 26859:20111013:163108.296 13: /lib/libc.so.6(strlen+0x33) [0x373453]
 26859:20111013:163108.296 12: /lib/libc.so.6(__strdup+0x25) [0x373195]
 26859:20111013:163108.296 11: /usr/local/sbin/zabbix_server(zbx_strdup2+0x2a) [0x808b79a]
 26859:20111013:163108.296 10: /usr/local/sbin/zabbix_server(DBget_history+0x2f2) [0x80a6192]
 26859:20111013:163108.296 9: /usr/local/sbin/zabbix_server(evaluate_function+0xc85) [0x8099035]
 26859:20111013:163108.296 8: /usr/local/sbin/zabbix_server(evaluate_expression+0x561) [0x809eda1]
 26859:20111013:163108.296 7: /usr/local/sbin/zabbix_server [0x807b2a1]
 26859:20111013:163108.296 6: /usr/local/sbin/zabbix_server(DCsync_history+0xf44) [0x807c394]
 26859:20111013:163108.296 5: /usr/local/sbin/zabbix_server(main_dbsyncer_loop+0xea) [0x805665a]
 26859:20111013:163108.296 4: /usr/local/sbin/zabbix_server(MAIN_ZABBIX_ENTRY+0xa2b) [0x8055e0b]
 26859:20111013:163108.296 3: /usr/local/sbin/zabbix_server(daemon_start+0x2b6) [0x8086086]
 26859:20111013:163108.296 2: /usr/local/sbin/zabbix_server(main+0x219) [0x8055229]
 26859:20111013:163108.296 1: /lib/libc.so.6(__libc_start_main+0xdc) [0x318e9c]
 26859:20111013:163108.296 0: /usr/local/sbin/zabbix_server [0x8051321]
 26859:20111013:163108.296 === Memory map: ===
 26859:20111013:163108.296 00110000-0023a000 r-xp 00000000 fd:00 265759     /lib/libcrypto.so.0.9.8e
 26859:20111013:163108.296 0023a000-0024d000 rwxp 00129000 fd:00 265759     /lib/libcrypto.so.0.9.8e
 26859:20111013:163108.296 0024d000-00251000 rwxp 0024d000 00:00 0 
 26859:20111013:163108.296 00251000-00258000 r-xp 00000000 fd:00 262385     /lib/libwrap.so.0.7.6
 26859:20111013:163108.296 00258000-00259000 rwxp 00007000 fd:00 262385     /lib/libwrap.so.0.7.6
 26859:20111013:163108.297 00259000-0026e000 r-xp 00000000 fd:00 265238     /lib/libnsl-2.5.so
 26859:20111013:163108.297 0026e000-0026f000 r-xp 00014000 fd:00 265238     /lib/libnsl-2.5.so
 26859:20111013:163108.297 0026f000-00270000 rwxp 00015000 fd:00 265238     /lib/libnsl-2.5.so
 26859:20111013:163108.297 00270000-00272000 rwxp 00270000 00:00 0 
 26859:20111013:163108.297 00272000-0027b000 r-xp 00000000 fd:00 265231     /lib/libcrypt-2.5.so
 26859:20111013:163108.297 0027b000-0027c000 r-xp 00008000 fd:00 265231     /lib/libcrypt-2.5.so
 26859:20111013:163108.297 0027c000-0027d000 rwxp 00009000 fd:00 265231     /lib/libcrypt-2.5.so
 26859:20111013:163108.297 0027d000-002a4000 rwxp 0027d000 00:00 0 
 26859:20111013:163108.297 002a4000-002ba000 r-xp 00000000 fd:00 262332     /lib/libselinux.so.1
 26859:20111013:163108.297 002ba000-002bc000 rwxp 00015000 fd:00 262332     /lib/libselinux.so.1
 26859:20111013:163108.297 002e4000-002ff000 r-xp 00000000 fd:00 262161     /lib/ld-2.5.so
 26859:20111013:163108.297 002ff000-00300000 r-xp 0001a000 fd:00 262161     /lib/ld-2.5.so
 26859:20111013:163108.297 00300000-00301000 rwxp 0001b000 fd:00 262161     /lib/ld-2.5.so
 26859:20111013:163108.297 00303000-00456000 r-xp 00000000 fd:00 262168     /lib/libc-2.5.so
 26859:20111013:163108.297 00456000-00458000 r-xp 00153000 fd:00 262168     /lib/libc-2.5.so
 26859:20111013:163108.297 00458000-00459000 rwxp 00155000 fd:00 262168     /lib/libc-2.5.so
 26859:20111013:163108.297 00459000-0045c000 rwxp 00459000 00:00 0 
 26859:20111013:163108.297 0045e000-00461000 r-xp 00000000 fd:00 262183     /lib/libdl-2.5.so
 26859:20111013:163108.297 00461000-00462000 r-xp 00002000 fd:00 262183     /lib/libdl-2.5.so
 26859:20111013:163108.297 00462000-00463000 rwxp 00003000 fd:00 262183     /lib/libdl-2.5.so
 26859:20111013:163108.297 00480000-00492000 r-xp 00000000 fd:00 262176     /lib/libz.so.1.2.3
 26859:20111013:163108.297 00492000-00493000 rwxp 00011000 fd:00 262176     /lib/libz.so.1.2.3
 26859:20111013:163108.297 00495000-004bc000 r-xp 00000000 fd:00 262181     /lib/libm-2.5.so
 26859:20111013:163108.297 004bc000-004bd000 r-xp 00026000 fd:00 262181     /lib/libm-2.5.so
 26859:20111013:163108.297 004bd000-004be000 rwxp 00027000 fd:00 262181     /lib/libm-2.5.so
 26859:20111013:163108.297 004c0000-00558000 r-xp 00000000 fd:00 1648514    /usr/lib/libnetsnmp.so.10.0.3
 26859:20111013:163108.297 00558000-0055a000 rwxp 00098000 fd:00 1648514    /usr/lib/libnetsnmp.so.10.0.3
 26859:20111013:163108.297 0055a000-00579000 rwxp 0055a000 00:00 0 
 26859:20111013:163108.298 00579000-005b4000 r-xp 00000000 fd:00 262234     /lib/libsepol.so.1
 26859:20111013:163108.298 005b4000-005b5000 rwxp 0003b000 fd:00 262234     /lib/libsepol.so.1
 26859:20111013:163108.298 005b5000-005bf000 rwxp 005b5000 00:00 0 
 26859:20111013:163108.298 0067e000-0067f000 r-xp 0067e000 00:00 0          [vdso]
 26859:20111013:163108.298 006e1000-006ec000 r-xp 00000000 fd:00 265234     /lib/libgcc_s-4.1.2-20080825.so.1
 26859:20111013:163108.298 006ec000-006ed000 rwxp 0000a000 fd:00 265234     /lib/libgcc_s-4.1.2-20080825.so.1
 26859:20111013:163108.298 006ef000-006ff000 r-xp 00000000 fd:00 265757     /lib/libresolv-2.5.so
 26859:20111013:163108.298 006ff000-00700000 r-xp 0000f000 fd:00 265757     /lib/libresolv-2.5.so
 26859:20111013:163108.298 00700000-00701000 rwxp 00010000 fd:00 265757     /lib/libresolv-2.5.so
 26859:20111013:163108.298 00701000-00703000 rwxp 00701000 00:00 0 
 26859:20111013:163108.298 00705000-00707000 r-xp 00000000 fd:00 265756     /lib/libkeyutils-1.2.so
 26859:20111013:163108.298 00707000-00708000 rwxp 00001000 fd:00 265756     /lib/libkeyutils-1.2.so
 26859:20111013:163108.298 0070a000-00712000 r-xp 00000000 fd:00 1648691    /usr/lib/libkrb5support.so.0.1
 26859:20111013:163108.298 00712000-00713000 rwxp 00007000 fd:00 1648691    /usr/lib/libkrb5support.so.0.1
 26859:20111013:163108.298 00715000-0073b000 r-xp 00000000 fd:00 1648792    /usr/lib/libk5crypto.so.3.1
 26859:20111013:163108.298 0073b000-0073c000 rwxp 00025000 fd:00 1648792    /usr/lib/libk5crypto.so.3.1
 26859:20111013:163108.298 0073e000-0076b000 r-xp 00000000 fd:00 1649411    /usr/lib/libgssapi_krb5.so.2.2
 26859:20111013:163108.298 0076b000-0076c000 rwxp 0002d000 fd:00 1649411    /usr/lib/libgssapi_krb5.so.2.2
 26859:20111013:163108.298 00781000-00783000 r-xp 00000000 fd:00 265758     /lib/libcom_err.so.2.1
 26859:20111013:163108.298 00783000-00784000 rwxp 00001000 fd:00 265758     /lib/libcom_err.so.2.1
 26859:20111013:163108.298 00873000-00907000 r-xp 00000000 fd:00 1649346    /usr/lib/libkrb5.so.3.3
 26859:20111013:163108.298 00907000-0090a000 rwxp 00093000 fd:00 1649346    /usr/lib/libkrb5.so.3.3
 26859:20111013:163108.298 0090c000-0093c000 r-xp 00000000 fd:00 1653482    /usr/lib/libidn.so.11.5.19
 26859:20111013:163108.298 0093c000-0093d000 rwxp 0002f000 fd:00 1653482    /usr/lib/libidn.so.11.5.19
 26859:20111013:163108.298 00957000-0099b000 r-xp 00000000 fd:00 265760     /lib/libssl.so.0.9.8e
 26859:20111013:163108.298 0099b000-0099f000 rwxp 00043000 fd:00 265760     /lib/libssl.so.0.9.8e
 26859:20111013:163108.298 009a1000-00ac2000 r-xp 00000000 fd:00 1867793    /usr/lib/mysql/libmysqlclient.so.15.0.0
 26859:20111013:163108.299 00ac2000-00b04000 rwxp 00120000 fd:00 1867793    /usr/lib/mysql/libmysqlclient.so.15.0.0
 26859:20111013:163108.299 00b04000-00b05000 rwxp 00b04000 00:00 0 
 26859:20111013:163108.299 00bcf000-00c0a000 r-xp 00000000 fd:00 1653580    /usr/lib/libcurl.so.3.0.0
 26859:20111013:163108.299 00c0a000-00c0b000 rwxp 0003b000 fd:00 1653580    /usr/lib/libcurl.so.3.0.0
 26859:20111013:163108.299 00cf3000-00cfd000 r-xp 00000000 fd:00 262228     /lib/libnss_files-2.5.so
 26859:20111013:163108.299 00cfd000-00cfe000 r-xp 00009000 fd:00 262228     /lib/libnss_files-2.5.so
 26859:20111013:163108.299 00cfe000-00cff000 rwxp 0000a000 fd:00 262228     /lib/libnss_files-2.5.so
 26859:20111013:163108.299 08048000-080cf000 r-xp 00000000 fd:00 295116     /usr/local/sbin/zabbix_server
 26859:20111013:163108.299 080cf000-080e8000 rw-p 00087000 fd:00 295116     /usr/local/sbin/zabbix_server
 26859:20111013:163108.299 080e8000-080ed000 rw-p 080e8000 00:00 0 
 26859:20111013:163108.299 09283000-092cb000 rw-p 09283000 00:00 0          [heap]
 26859:20111013:163108.299 b5b61000-b5b64000 rw-s 00000000 00:09 4947979    /SYSV530085ab (deleted)
 26859:20111013:163108.299 b5b64000-b5c98000 rw-s 00000000 00:09 4915210    /SYSV730085ab (deleted)
 26859:20111013:163108.299 b5c98000-b6365000 rw-s 00000000 00:09 4882441    /SYSV670085ab (deleted)
 26859:20111013:163108.299 b6365000-b6766000 rw-s 00000000 00:09 4849672    /SYSV740085ab (deleted)
 26859:20111013:163108.299 b6766000-b7767000 rw-s 00000000 00:09 4816903    /SYSV780085ab (deleted)
 26859:20111013:163108.299 b7767000-b7f6f000 rw-s 00000000 00:09 4784134    /SYSV680085ab (deleted)
 26859:20111013:163108.299 b7f6f000-b7f75000 rw-p b7f6f000 00:00 0 
 26859:20111013:163108.299 b7f7a000-b7f7c000 rw-p b7f7a000 00:00 0 
 26859:20111013:163108.299 bfc56000-bfc7d000 rw-p bffd7000 00:00 0          [stack]
 26859:20111013:163108.299 ================================
 26859:20111013:163108.299 Please consider attaching a disassembly listing to your bug report.
 26859:20111013:163108.299 This listing can be produced with, e.g., objdump -DSswx zabbix_server.
 26859:20111013:163108.299 ================================
 26789:20111013:163108.301 One child process died (PID:26859,exitcode/signal:255). Exiting ...
 26789:20111013:163108.301 zbx_on_exit() called
 26796:20111013:163108.301 Got signal [signal:15(SIGTERM),sender_pid:26789,sender_uid:500,reason:0]. Exiting ...
 26797:20111013:163108.302 Got signal [signal:15(SIGTERM),sender_pid:26789,sender_uid:500,reason:0]. Exiting ...
 26792:20111013:163108.303 Got signal [signal:15(SIGTERM),sender_pid:26789,sender_uid:500,reason:0]. Exiting ...
 26798:20111013:163108.304 Got signal [signal:15(SIGTERM),sender_pid:26789,sender_uid:500,reason:0]. Exiting ...
 26794:20111013:163108.305 Got signal [signal:15(SIGTERM),sender_pid:26789,sender_uid:500,reason:0]. Exiting ...
 26800:20111013:163108.305 Got signal [signal:15(SIGTERM),sender_pid:26789,sender_uid:500,reason:0]. Exiting ...
 26801:20111013:163108.307 Got signal [signal:15(SIGTERM),sender_pid:26789,sender_uid:500,reason:0]. Exiting ...
26854:20111013:163108.314 Syncing ...
......
    Tags: None
    • LPby
      Junior Member
      • Aug 2008
      • 21
      #2
      Проблема была в том, что один из элементов данных в новой базе почему-то отсутствовал. Поиск в таблице history по itemid=22265 (перед падением был поиск в БД именно этого элемента) ничего не вернул. Удаление этого элемента и повторное создание решило проблему))
      Last edited by LPby; 13-10-2011, 15:32.

        Comment

        • wax
          Junior Member
          • Sep 2011
          • 16
          #3
          Доброго времени суток.

          Вот тут ( http://www.zabbix.com/forum/showthread.php?p=91394 ) столкнулся с такой же проблеммой под фряхой. Только не смог определить, где и чего не хватает. Посоветуйте , пожалуйста, алгоритм поиска граблей.

          PS: может стоит создать багрепорт в здешней jira? Не айс же, если система мониторинга падает от такой мелочи...

            Comment

            Previous template Next
            Working...