Ad Widget

Collapse

*** buffer overflow detected *** terminated

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Siv
    Junior Member
    • Feb 2023
    • 4
    #1

    *** buffer overflow detected *** terminated

    Добрый день !
    ребята помогите
    установил на ubuntu 22.04 Zabbix 6.2.7 (revision d7e8df54d76).
    ювалятся одибки по 10 шт. в минуту причем рандомно , включал отладку пытался отловить причину он оа не очевидна
    *** buffer overflow detected *** terminated
    zabbix сервис постоянно падаи и поднимается иногда не поднимается лприходится убивать остатки пройцессов
    докопался что возможно проблема в glibc 2.35-5 но как победить с ходу не пойму .
    буду благодарен за помощь , возможно у когото встречалось
    сильно странно что с коробки так происходит
    Tags: None
    • Siv
      Junior Member
      • Feb 2023
      • 4
      #2
      простите писал с телефона вагон ошибок но править у меня прав не хватает
      нибе исправлено

      ------
      Добрый день !
      ребята помогите
      установил на ubuntu 22.04 Zabbix 6.2.7 (revision d7e8df54d76).
      валятся ошибки по 10 шт. в минуту причем рандомно , включал отладку пытался отловить причину он оа не очевидна
      *** buffer overflow detected *** terminated
      zabbix сервис постоянно падает и поднимается иногда не поднимается приходится убивать остатки процессов
      докопался что возможно проблема в glibc 2.35-5 но как победить с ходу не пойму .
      буду благодарен за помощь , возможно у кого то встречалось
      сильно странно что с коробки так происходит
      ​

        Comment

        • Glencoe
          Zabbix developer
          • Oct 2019
          • 152
          #3
          Приведите пож-та полный лог креша, всё что вокруг слова Backtrace. Это поможет понять в какую сторону копать.

            Comment

            • Siv
              Junior Member
              • Feb 2023
              • 4
              #4
              у меня в логах нет Backtrace
              выглядит все както мало информативно
              это только один из кусков
              ....
              JOIN (SELECT cntr_value,OBJECT_NAME
              FROM sys.dm_os_performance_counters
              WHERE counter_name='Cache Hit Ratio base' AND OBJECT_NAME='SQLServer:Plan Cache' AND instance_name='_Total') b
              ON a.OBJECT_NAME=b.OBJECT_NAME
              WHERE a.counter_name='Cache Hit Ratio' AND a.OBJECT_NAME='SQLServer:Plan Cache' AND instance_name='_Total''
              121528:20230206:153500.339 In zbx_odbc_connect() dsn:'sql2c.clv.ru' user:'zbx_monitor'
              121514:20230206:153500.348 zbx_setproctitle() title:'alert syncer [queuing alerts]'
              121514:20230206:153500.348 In am_db_get_alerts()
              121514:20230206:153500.348 query [txnlev:1] [begin;]
              121514:20230206:153500.348 query [txnlev:1] [select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a.p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid]
              *** buffer overflow detected ***: terminated
              121514:20230206:153500.348 query [txnlev:1] [commit;]
              121514:20230206:153500.348 End of am_db_get_alerts():SUCCEED alerts:0
              121514:20230206:153500.348 In am_db_flush_results()
              121514:20230206:153500.348 In zbx_ipc_socket_write()
              121514:20230206:153500.348 End of zbx_ipc_socket_write():SUCCEED
              121514:20230206:153500.348 In zbx_ipc_socket_read()
              121474:20230206:153500.348 End of zbx_ipc_service_recv():1
              121474:20230206:153500.348 In am_flush_results() results:0
              121474:20230206:153500.348 In zbx_ipc_client_send() clientid:6
              121474:20230206:153500.348 End of zbx_ipc_client_send():SUCCEED
              121514:20230206:153500.348 End of zbx_ipc_socket_read():SUCCEED
              121514:20230206:153500.348 End of am_db_flush_results() flushed:0
              121514:20230206:153500.348 zbx_setproctitle() title:'alert syncer [queued 0 alerts(s), flushed 0 result(s) in 0.000670 sec, idle 1 sec]'
              121474:20230206:153500.349 End of am_flush_results()
              121474:20230206:153500.349 In zbx_ipc_service_recv() timeout:1.000
              121521:20230206:153500.394 End of zbx_ipc_async_socket_recv():0
              121524:20230206:153500.394 End of zbx_ipc_async_socket_recv():0
              121521:20230206:153500.394 zbx_setproctitle() title:'history poller Forum [got 1 values in 0.002346 sec, getting values]'
              121521:20230206:153500.394 In get_values()
              121521:20230206:153500.394 In DCconfig_get_poller_items() poller_type:5
              121524:20230206:153500.394 zbx_setproctitle() title:'history poller #5 [got 1 values in 0.002587 sec, getting values]'
              121524:20230206:153500.394 In get_values()
              121524:20230206:153500.394 In DCconfig_get_poller_items() poller_type:5
              121521:20230206:153500.394 End of DCconfig_get_poller_items():1​
              ....
              Last edited by Siv; 06-02-2023, 15:08.

                Comment

                • vso
                  Zabbix developer
                  • Aug 2016
                  • 190
                  #5
                  Это скорее всего проблема с каким-то ODBC драйвером, описано тут:
                  https://www.zabbix.com/documentation...n/known_issues
                  Попробуйте отключить ODBC проверки

                    Comment

                    • Siv
                      Junior Member
                      • Feb 2023
                      • 4
                      #6
                      @VCO
                      ODBC это первое о чем я подумал исключил отовсюду MSSQL шаблон
                      но не нашел метода как явно найти на чем валится переполнение буфера . например ка это обьяснить ?



                      229713:20230207:055850.365 End of get_value_agent():SUCCEED
                      229713:20230207:055850.365 End of get_value():SUCCEED
                      229713:20230207:055850.365 In zbx_activate_item_interface() interfaceid:21 itemid:45890 type:0
                      229713:20230207:055850.365 End of zbx_activate_item_interface()
                      229713:20230207:055850.365 In zbx_preprocess_item_value()
                      229718:20230207:055850.365 get value from agent result: '0'
                      229713:20230207:055850.365 End of zbx_preprocess_item_value()
                      229718:20230207:055850.365 End of get_value_agent():SUCCEED
                      229718:20230207:055850.365 End of get_value():SUCCEED
                      229718:20230207:055850.365 In zbx_activate_item_interface() interfaceid:99 itemid:47810 type:0
                      229718:20230207:055850.365 End of zbx_activate_item_interface()
                      229718:20230207:055850.365 In zbx_preprocess_item_value()
                      229718:20230207:055850.365 End of zbx_preprocess_item_value()
                      229713:20230207:055850.365 In zbx_ipc_socket_write()
                      229718:20230207:055850.365 In zbx_ipc_socket_write()
                      229718:20230207:055850.365 End of zbx_ipc_socket_write():SUCCEED
                      229718:20230207:055850.365 End of get_values():1
                      229718:20230207:055850.365 zbx_setproctitle() title:'poller #5 [got 7 values in 0.110685 sec, idle 1 sec]'
                      229718:20230207:055850.365 In zbx_ipc_async_socket_recv() timeout:1
                      229692:20230207:055850.365 End of zbx_ipc_service_recv():1
                      229692:20230207:055850.365 In preprocessor_add_request()
                      229692:20230207:055850.365 In preprocessor_sync_configuration()
                      229692:20230207:055850.365 End of preprocessor_sync_configuration() item config size: 666, history cache size: 123
                      229692:20230207:055850.365 In preprocessor_enqueue() itemid: 45890
                      229692:20230207:055850.365 In preprocessor_enqueue_dependent() itemid: 45890
                      229692:20230207:055850.365 End of preprocessor_enqueue_dependent()
                      229692:20230207:055850.365 End of preprocessor_enqueue()
                      229692:20230207:055850.365 In preprocessor_assign_tasks()
                      229692:20230207:055850.365 In preprocessor_get_next_task()
                      229692:20230207:055850.365 End of preprocessor_get_next_task()
                      229692:20230207:055850.365 End of preprocessor_assign_tasks()
                      229692:20230207:055850.365 End of preprocessor_add_request()
                      229713:20230207:055850.365 End of zbx_ipc_socket_write():SUCCEED
                      229713:20230207:055850.365 End of get_values():1
                      229713:20230207:055850.365 zbx_setproctitle() title:'poller Forum [got 5 values in 0.095174 sec, idle 1 sec]'
                      229713:20230207:055850.365 In zbx_ipc_async_socket_recv() timeout:1
                      229692:20230207:055850.365 In zbx_ipc_service_recv() timeout:1.000
                      229692:20230207:055850.365 End of zbx_ipc_service_recv():1
                      229692:20230207:055850.365 In preprocessor_add_request()
                      229692:20230207:055850.365 In preprocessor_sync_configuration()
                      229692:20230207:055850.365 End of preprocessor_sync_configuration() item config size: 666, history cache size: 123
                      229692:20230207:055850.365 In preprocessor_enqueue() itemid: 47810
                      229692:20230207:055850.365 In preprocessor_enqueue_dependent() itemid: 47810
                      229692:20230207:055850.365 End of preprocessor_enqueue_dependent()
                      229692:20230207:055850.365 End of preprocessor_enqueue()
                      229692:20230207:055850.365 In preprocessor_assign_tasks()
                      229692:20230207:055850.365 In preprocessor_get_next_task()
                      229692:20230207:055850.365 End of preprocessor_get_next_task()
                      229692:20230207:055850.365 End of preprocessor_assign_tasks()
                      229692:20230207:055850.365 End of preprocessor_add_request()
                      229692:20230207:055850.365 In zbx_ipc_service_recv() timeout:1.000
                      229688:20230207:055850.367 End of zbx_ipc_service_recv():2
                      229726:20230207:055850.367 zbx_setproctitle() title:'alert syncer [queuing alerts]'
                      229726:20230207:055850.367 In am_db_get_alerts()
                      229688:20230207:055850.367 In zbx_ipc_service_recv() timeout:1.000
                      229726:20230207:055850.367 query [txnlev:1] [begin;]
                      229726:20230207:055850.367 query [txnlev:1] [select a.alertid,a.mediatypeid,a.sendto,a.subject,a.messa ge,a.status,a.retries,e.source,e.object,e.objectid ,a.parameters,a.eventid,a.p_eventid from alerts a left join events e on a.eventid=e.eventid where alerttype=0 and a.status=3 order by a.alertid]
                      229726:20230207:055850.367 query [txnlev:1] [commit;]
                      229726:20230207:055850.367 End of am_db_get_alerts():SUCCEED alerts:0
                      229726:20230207:055850.367 In am_db_flush_results()
                      229726:20230207:055850.367 In zbx_ipc_socket_write()
                      229726:20230207:055850.368 End of zbx_ipc_socket_write():SUCCEED
                      229726:20230207:055850.368 In zbx_ipc_socket_read()
                      229688:20230207:055850.368 End of zbx_ipc_service_recv():1
                      229688:20230207:055850.368 In am_flush_results() results:0
                      229688:20230207:055850.368 In zbx_ipc_client_send() clientid:6
                      229688:20230207:055850.368 End of zbx_ipc_client_send():SUCCEED
                      229688:20230207:055850.368 End of am_flush_results()
                      229688:20230207:055850.368 In zbx_ipc_service_recv() timeout:1.000
                      229726:20230207:055850.368 End of zbx_ipc_socket_read():SUCCEED
                      229726:20230207:055850.368 End of am_db_flush_results() flushed:0
                      229726:20230207:055850.368 zbx_setproctitle() title:'alert syncer [queued 0 alerts(s), flushed 0 result(s) in 0.000618 sec, idle 1 sec]'
                      *** buffer overflow detected ***: terminated
                      229684:20230207:055850.443 One child process died (PID:229744,exitcode/signal:6). Exiting ...
                      229684:20230207:055850.443 End of zbx_ipc_service_recv():1
                      229684:20230207:055850.443 In zbx_ha_dispatch_message()
                      229684:20230207:055850.443 In zbx_ipc_socket_close()
                      229684:20230207:055850.443 End of zbx_ipc_socket_close()
                      229684:20230207:055850.443 PROCESS EXIT: 229744​

                        Comment

                        • vso
                          Zabbix developer
                          • Aug 2016
                          • 190
                          #7
                          Нужны последние логи для процесса 229744
                          229684:20230207:055850.443 One child process died (PID:229744,exitcode/signal:6). Exiting ...

                            Comment

                            • ilyavantonov
                              Junior Member
                              • Feb 2023
                              • 1
                              #8

                              Добрый день !
                              Аналогичная проблема, проявляется с использованием шаблона 'MSSQL by ODBC'.

                              ОС: Ubuntu 22.04
                              Zabbix-server-pgsql - 6.2.7-1+ubuntu22.04

                              ODBC: msodbcsql17 (пробовал и 18ю версию - дело не в этом)

                              Со стороны MSSQL:
                              Windows Server 2019 Standart
                              MSSQL 2019

                              в логе сначала запись о смерти процесса:
                              Code:
                              23390:20230227:165300.865 One child process died (PID:23538,exitcode/signal:6). Exiting ...
                              который является odbc poller:
                              Code:
                              23538:20230227:165215.648 server #112 started [odbc poller #8]]
                              и "падает" Zabbix после получения большой "простыни" из MSSQL по запросу:
                              Code:
                               23538:20230227:165300.684 In get_value_db() key_orig:'db.odbc.get[get_status_variables,"{$MSSQL.DSN}"]' query:'SELECT object_name,counter_name,instance_name,cntr_value
FROM sys.dm_os_performance_counters
UNION
SELECT 'Live-SQL-Serv' as object_name,'Version' as counter_name,@@version as instance_name,0 as cntr_value
UNION
SELECT 'Live-SQL-Serv' as object_name,'Uptime' as counter_name,'' as instance_name,DATEDIFF(second,sqlserver_start_time,GETDATE()) as cntr_value
FROM sys.dm_os_sys_info
UNION
SELECT 'Live-SQL-Serv:Databases' as object_name,'State' as counter_name,name as instance_name,state as cntr_value
FROM sys.databases
UNION
SELECT a.object_name,'BufferCacheHitRatio' as counter_name,'' as instance_name,cast(a.cntr_value*100.0/b.cntr_value as dec(3,0)) as cntr_value
FROM sys.dm_os_performance_counters a
JOIN (SELECT cntr_value,OBJECT_NAME
FROM sys.dm_os_performance_counters
WHERE counter_name='Buffer cache hit ratio base' AND OBJECT_NAME='Live-SQL-Serv:Buffer Manager') b
ON a.OBJECT_NAME=b.OBJECT_NAME
WHERE a.counter_name='Buffer cache hit ratio' AND a.OBJECT_NAME='Live-SQL-Serv:Buffer Manager'
UNION
SELECT a.object_name,'WorktablesFromCacheRatio' as counter_name,'' as instance_name,cast(a.cntr_value*100.0/b.cntr_value as dec(3,0)) as cntr_value
FROM sys.dm_os_performance_counters a
JOIN (SELECT cntr_value,OBJECT_NAME
FROM sys.dm_os_performance_counters
WHERE counter_name='Worktables From Cache Base' AND OBJECT_NAME='Live-SQL-Serv:Access Methods') b
ON a.OBJECT_NAME=b.OBJECT_NAME
WHERE a.counter_name='Worktables From Cache Ratio' AND a.OBJECT_NAME='Live-SQL-Serv:Access Methods'
UNION
SELECT a.object_name,'CacheHitRatio' as counter_name,'_Total' as instance_name,cast(a.cntr_value*100.0/b.cntr_value as dec(3,0)) as cntr_value
FROM sys.dm_os_performance_counters a
JOIN (SELECT cntr_value,OBJECT_NAME
FROM sys.dm_os_performance_counters
WHERE counter_name='Cache Hit Ratio base' AND OBJECT_NAME='Live-SQL-Serv:Plan Cache' AND instance_name='_Total') b
ON a.OBJECT_NAME=b.OBJECT_NAME
WHERE a.counter_name='Cache Hit Ratio' AND a.OBJECT_NAME='Live-SQL-Serv:Plan Cache' AND instance_name='_Total''
                              ​
                              Далее происходит переполнение буфера:

                              Code:
                              ​​​​​​ 23538:20230227:165300.684 In zbx_odbc_connect() dsn:'Live-SQL-Serv' user:'zbx_monitor'
23538:20230227:165300.685 zbx_odbc_diag(): [SQL_SUCCESS]
23538:20230227:165300.685 zbx_odbc_diag(): [SQL_SUCCESS]
23538:20230227:165300.685 zbx_odbc_diag(): [SQL_SUCCESS]
*** buffer overflow detected ***: terminated
                              ​
                              Лог с дебагом слишком большой (даже в архиве) - выложил в облако

                              Так же прошу помочь в решении данной проблемы.

                                Comment

                                Previous template Next
                                Working...