Ad Widget

Collapse

Get value from agent failed: Cannot establish TCP connection

Collapse
This topic has been answered.
X
X
 
  • Time
  • Show
Clear All
new posts
  • whanklee
    Junior Member
    • Dec 2019
    • 19

    #1

    Get value from agent failed: Cannot establish TCP connection

    Hi.

    After upgrading to 7.0 from 6.4 Zabbix_agent doesn't work. It worked well before upgrade.

    Agent log:

    Code:
    640432:20240612:190932.452 Starting Zabbix Agent [agent-server-host]. Zabbix 7.0.0 (revision 49955f1fb5c).
    
    640432:20240612:190932.452 **** Enabled features ****
    
    640432:20240612:190932.452 IPv6 support:          YES
    
    640432:20240612:190932.452 TLS support:           YES
    
    640432:20240612:190932.452 **************************
    
    640432:20240612:190932.452 using configuration file: /etc/zabbix/zabbix_agentd.conf
    
    640432:20240612:190932.454 agent #0 started [main process]
    
    640433:20240612:190932.455 agent #1 started [collector]
    
    640434:20240612:190932.455 agent #2 started[listener #1]
    
    640435:20240612:190932.456 agent #3 started[listener #2]
    
    640436:20240612:190932.456 agent #4 started[listener #3]
    
    640438:20240612:190932.460 agent #6 started[listener #5]
    
    640437:20240612:190932.464 agent #5 started[listener #4]
    
    640439:20240612:190932.465 agent #7 started[listener #6]
    
    640441:20240612:190932.468 agent #9 started[listener #8]
    
    640440:20240612:190932.468 agent #8 started[listener #7]
    
    640443:20240612:190932.469 agent #11 started[listener #10]
    
    640444:20240612:190932.472 agent #12 started [active checks #1]
    
    640442:20240612:190932.472 agent #10 started[listener #9]
    
    640433:20240612:190933.456 Got signal [signal:11(SIGSEGV),reason:1,refaddr:(nil)]. Crashing ...
    
    640433:20240612:190933.456 ====== Fatal information: ======
    
    640433:20240612:190933.456 Program counter: 0x5584d6e884d0
    
    640433:20240612:190933.456 === Registers: ===
    
    640433:20240612:190933.456 r8      =     5584d76350b0 =       94029037654192 =       94029037654192
    
    640433:20240612:190933.456 r9      =                0 =                    0 =                    0
    
    640433:20240612:190933.456 r10     =     7f1d7fc9eac0 =      139764674718400 =      139764674718400
    
    640433:20240612:190933.456 r11     =              246 =                  582 =                  582
    
    640433:20240612:190933.456 r12     =                0 =                    0 =                    0
    
    640433:20240612:190933.456 r13     =     7ffc10bdb070 =      140720589353072 =      140720589353072
    
    640433:20240612:190933.456 r14     =                0 =                    0 =                    0
    
    640433:20240612:190933.456 r15     =     5584d76350b0 =       94029037654192 =       94029037654192
    
    640433:20240612:190933.456 rdi     =           540036 =              5505078 =              5505078
    
    640433:20240612:190933.456 rsi     =                0 =                    0 =                    0
    
    640433:20240612:190933.456 rbp     =                0 =                    0 =                    0
    
    640433:20240612:190933.456 rbx     =     5584d6ee1051 =       94029029970001 =       94029029970001
    
    640433:20240612:190933.456 rdx     =                0 =                    0 =                    0
    
    640433:20240612:190933.456 rax     =                8 =                    8 =                    8
    
    640433:20240612:190933.456 rcx     =     7f1d7fc083eb =      139764674102251 =      139764674102251
    
    640433:20240612:190933.456 rsp     =     7ffc10bdb060 =      140720589353056 =      140720589353056
    
    640433:20240612:190933.456 rip     =     5584d6e884d0 =       94029029606608 =       94029029606608
    
    640433:20240612:190933.456 efl     =            10206 =                66054 =                66054
    
    640433:20240612:190933.456 csgsfs  =   2b000000000033 =    12103423998558259 =    12103423998558259
    
    640433:20240612:190933.456 err     =                4 =                    4 =                    4
    
    640433:20240612:190933.456 trapno  =                e =                   14 =                   14
    
    640433:20240612:190933.456 oldmask =                0 =                    0 =                    0
    
    640433:20240612:190933.456 cr2     =                0 =                    0 =                    0
    
    640433:20240612:190933.456 === Backtrace: ===
    
    640433:20240612:190933.457 11: /usr/sbin/zabbix_agentd: collector [processing data](zbx_backtrace+0x52) [0x5584d6ea4482]
    
    640433:20240612:190933.457 10: /usr/sbin/zabbix_agentd: collector [processing data](zbx_log_fatal_info+0x145) [0x5584d6ea46c5]
    
    640433:20240612:190933.457 9: /usr/sbin/zabbix_agentd: collector [processing data](+0x58f2a) [0x5584d6ea4f2a]
    
    640433:20240612:190933.458 8: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f1d7fb22520]
    
    640433:20240612:190933.458 7: /usr/sbin/zabbix_agentd: collector [processing data](collect_stats_diskdevices+0x40) [0x5584d6e884d0]
    
    640433:20240612:190933.458 6: /usr/sbin/zabbix_agentd: collector [processing data](zbx_collector_thread+0xc3) [0x5584d6e86dd3]
    
    640433:20240612:190933.458 5: /usr/sbin/zabbix_agentd: collector [processing data](zbx_thread_start+0x24) [0x5584d6ea1614]
    
    640433:20240612:190933.458 4: /usr/sbin/zabbix_agentd: collector [processing data](MAIN_ZABBIX_ENTRY+0x52d) [0x5584d6e69add]
    
    640433:20240612:190933.458 3: /usr/sbin/zabbix_agentd: collector [processing data](main+0x33c) [0x5584d6e679dc]
    
    640433:20240612:190933.458 2: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f1d7fb09d90]
    
    640433:20240612:190933.458 1: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f1d7fb09e40]
    
    640433:20240612:190933.458 0: /usr/sbin/zabbix_agentd: collector [processing data](_start+0x25) [0x5584d6e67a85]
    
    640433:20240612:190933.458 === Memory map: ===
    
    640433:20240612:190933.458 5584d6e4c000-5584d6e65000 r--p 00000000 fc:02 5432                       /usr/sbin/zabbix_agentd
    
    640433:20240612:190933.458 5584d6e65000-5584d6edb000 r-xp 00019000 fc:02 5432                       /usr/sbin/zabbix_agentd
    
    640433:20240612:190933.458 5584d6edb000-5584d6f03000 r--p 0008f000 fc:02 5432                       /usr/sbin/zabbix_agentd
    
    640433:20240612:190933.458 5584d6f03000-5584d6f05000 r--p 000b6000 fc:02 5432                       /usr/sbin/zabbix_agentd
    
    ....
    640433:20240612:190933.463 7ffc10bbd000-7ffc10bde000 rw-p 00000000 00:00 0                          [stack]
    
    640433:20240612:190933.463 7ffc10bed000-7ffc10bf1000 r--p 00000000 00:00 0                          [vvar]
    
    640433:20240612:190933.464 7ffc10bf1000-7ffc10bf3000 r-xp 00000000 00:00 0                          [vdso]
    
    640433:20240612:190933.464 ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]
    
    640433:20240612:190933.464 ================================
    
    640433:20240612:190933.464 Please consider attaching a disassembly listing to your bug report.
    
    640433:20240612:190933.464 This listing can be produced with, e.g., objdump -DSswx zabbix_agentd.
    
    640433:20240612:190933.464 ================================
    
    640432:20240612:190933.465 One child process died (PID:640433,exitcode/signal:1). Exiting ...
    
    zabbix_agentd [640432]: Error waiting for process with PID 640433: [10] No child processes
    
    640432:20240612:190933.467 Zabbix Agent stopped. Zabbix 7.0.0 (revision 49955f1fb5c).
    Agent Configuration:

    Code:
    PidFile=/run/zabbix/zabbix_agentd.pid
    LogFile=/var/log/zabbix/zabbix_agentd.log
    LogFileSize=0
    Server=zabbix-server-host,127.0.0.1
    ServerActive=zabbix-server-host
    Hostname=agent-server-host
    Timeout=30
    Include=/etc/zabbix/zabbix_agentd.d/*.conf
    TLSConnect=psk
    TLSAccept=psk
    TLSPSKIdentity=PSK 002
    TLSPSKFile=/etc/zabbix/zabbix_agentd.psk
    Code:
    
    systemctl status zabbix-agent.service
    
    ● zabbix-agent.service - Zabbix Agent
    
         Loaded: loaded (/lib/systemd/system/zabbix-agent.service; enabled; vendor preset: enabled)
    
         Active: activating (auto-restart) (Result: exit-code) since Wed 2024-06-12 19:26:48 CEST; 9s ago
    
        Process: 647656 ExecStart=/usr/sbin/zabbix_agentd -c $CONFFILE (code=exited, status=0/SUCCESS)
    
        Process: 647713 ExecStop=/bin/sh -c [ -n "$1" ] && kill -s TERM "$1" -- $MAINPID [B](code=exited, status=1/FAILURE)[/B]
    
       Main PID: 647658 (code=exited, status=0/SUCCESS)
    
            CPU: 224ms
    Configuration on web:

    Click image for larger version  Name:	1.jpg Views:	0 Size:	256.6 KB ID:	485523

    Click image for larger version  Name:	2.jpg Views:	0 Size:	100.2 KB ID:	485524

    Click image for larger version

Name:	1.jpg
Views:	12267
Size:	105.7 KB
ID:	485564

    FYI: Zabbix Agent runs, if Zabbix Server is stopped.
    Attached Files
    Last edited by whanklee; 13-06-2024, 09:55.
  • Answer selected by whanklee at 16-06-2024, 18:03.
    whanklee
    Junior Member
    • Dec 2019
    • 19

    I can confirme that Zabbix Agent 6.4.1 works well. There are sose bugs in Zabbix Agent 7.0.x
    My problem is solved.

    Comment

    • whanklee
      Junior Member
      • Dec 2019
      • 19

      #2
      Itried something else. Turned off PSK on Server Host Configuration and in Agent.conf.
      I got new error:

      Click image for larger version  Name:	3.jpg Views:	0 Size:	154.2 KB ID:	485567

      Log Line:
      1017481:20240613:100027.767 failed to accept an incoming connection: from xxxxxxxxx: TLS connections are not allowed
      Last edited by whanklee; 13-06-2024, 10:02.

      Comment

      • Markku
        Senior Member
        Zabbix Certified SpecialistZabbix Certified ProfessionalZabbix Certified Expert
        • Sep 2018
        • 1781

        #3
        What's the deal with the agent hostname, am I missing something? In the first post screenshot there is "agent-host-server:10050", in the second post it is "agent-server-host:10050"

        Markku

        Comment

      • whanklee
        Junior Member
        • Dec 2019
        • 19

        #4
        I upgraded Zabbix from version 6.4 to 7.0. The active and passive Agent worked with the previous version, but since upgrading I always receive an error and I am unable to start Agent.

        Architecture:
        • Server A named Zabbix server (DNS ultron.xxxxxx.xxx): Zabbix server and 1 active and passive Zabbix Agent
        • Server B named Thanos (DNS thanos.xxxxxx.xxx): 1 active and passive Zabbix Agent
        • Operation system version Ubuntu 22.04
        • PostgreSQL 16

        The Agent installed on the Zabbix server is working, and there is no any problems on that server; both active and passive checks are working well.

        However, on 'Server B' named Thanos, only the active check works. The Zabbix server cannot reach the Agent on port 10050 because it constantly stops. Interestingly, if I stop the Zabbix server and restart the Agent on Thanos, it starts, but as soon as the Zabbix server is restarted, the Agent immediately stops with an error code.

        I did not change the configuration after the upgrade; it worked with this configuration previously. I cannot figure out what might be causing the error.

        Click image for larger version

Name:	get-value-from-agent-failed-cannot-establish-tcp-connection-v0-tj0nce54iq6d1.webp
Views:	12301
Size:	31.9 KB
ID:	485725

        /etc/zabbix/zabbix_agentd.conf:
        Code:
        PidFile=/run/zabbix/zabbix_agentd.pid
        LogFile=/var/log/zabbix/zabbix_agentd.log
        LogFileSize=0
        Server=127.0.0.1,ultron.xxxxx.xx
        ServerActive=ultron.xxxxx.xx
        Hostname=thanos.xxxxx.xx
        Include=/etc/zabbix/zabbix_agentd.d/*.conf
        TLSConnect=psk
        TLSAccept=psk
        TLSPSKIdentity=PSK 002
        TLSPSKFile=/etc/zabbix/zabbix_agentd.psk
        less /var/log/zabbix/zabbix_agentd.log
        Code:
        2229259:20240615:150123.526 Starting Zabbix Agent [thanos.xxxxxx.xx]. Zabbix 7.0.0 (revision 49955f1fb5c).
        2229259:20240615:150123.526 **** Enabled features ****
        2229259:20240615:150123.526 IPv6 support: YES
        2229259:20240615:150123.526 TLS support: YES
        2229259:20240615:150123.526 **************************
        2229259:20240615:150123.526 using configuration file: /etc/zabbix/zabbix_agentd.conf
        2229259:20240615:150123.528 agent #0 started [main process]
        2229260:20240615:150123.529 agent #1 started [collector]
        2229261:20240615:150123.535 agent #2 started[listener #1]
        2229263:20240615:150123.537 agent #4 started[listener #3]
        2229271:20240615:150123.539 agent #12 started [active checks #1]
        2229265:20240615:150123.541 agent #6 started[listener #5]
        2229264:20240615:150123.541 agent #5 started[listener #4]
        2229267:20240615:150123.545 agent #8 started[listener #7]
        2229268:20240615:150123.547 agent #9 started[listener #8]
        2229266:20240615:150123.549 agent #7 started[listener #6]
        2229269:20240615:150123.553 agent #10 started[listener #9]
        2229262:20240615:150123.553 agent #3 started[listener #2]
        2229270:20240615:150123.561 agent #11 started[listener #10]
        2229260:20240615:150124.530 Got signal [signal:11(SIGSEGV),reason:1,refaddr:(nil)]. Crashing ...
        2229260:20240615:150124.530 ====== Fatal information: ======
        2229260:20240615:150124.530 Program counter: 0x5629c04224d0
        2229260:20240615:150124.530 === Registers: ===
        2229260:20240615:150124.530 r8 = 5629c14b90c0 = 94737336602816 = 94737336602816
        2229260:20240615:150124.530 r9 = 0 = 0 = 0
        2229260:20240615:150124.530 r10 = 7f211afcfac0 = 139780163435200 = 139780163435200
        2229260:20240615:150124.530 r11 = 246 = 582 = 582
        2229260:20240615:150124.530 r12 = 0 = 0 = 0
        2229260:20240615:150124.530 r13 = 7ffefda059f0 = 140733153565168 = 140733153565168
        2229260:20240615:150124.530 r14 = 0 = 0 = 0
        2229260:20240615:150124.530 r15 = 5629c14b90c0 = 94737336602816 = 94737336602816
        2229260:20240615:150124.530 rdi = 1a40032 = 27525170 = 27525170
        2229260:20240615:150124.531 rsi = 0 = 0 = 0
        2229260:20240615:150124.531 rbp = 0 = 0 = 0
        2229260:20240615:150124.531 rbx = 5629c047b051 = 94737319571537 = 94737319571537
        2229260:20240615:150124.531 rdx = 0 = 0 = 0
        2229260:20240615:150124.531 rax = 8 = 8 = 8
        2229260:20240615:150124.531 rcx = 7f211af393eb = 139780162819051 = 139780162819051
        2229260:20240615:150124.531 rsp = 7ffefda059e0 = 140733153565152 = 140733153565152
        2229260:20240615:150124.531 rip = 5629c04224d0 = 94737319208144 = 94737319208144
        2229260:20240615:150124.531 efl = 10202 = 66050 = 66050
        2229260:20240615:150124.531 csgsfs = 2b000000000033 = 12103423998558259 = 12103423998558259
        2229260:20240615:150124.531 err = 4 = 4 = 4
        2229260:20240615:150124.531 trapno = e = 14 = 14
        2229260:20240615:150124.531 oldmask = 0 = 0 = 0
        2229260:20240615:150124.531 cr2 = 0 = 0 = 0
        2229260:20240615:150124.531 === Backtrace: ===
        2229260:20240615:150124.532 11: /usr/sbin/zabbix_agentd: collector [processing data](zbx_backtrace+0x52) [0x5629c043e482]
        2229260:20240615:150124.532 10: /usr/sbin/zabbix_agentd: collector [processing data](zbx_log_fatal_info+0x145) [0x5629c043e6c5]
        2229260:20240615:150124.532 9: /usr/sbin/zabbix_agentd: collector [processing data](+0x58f2a) [0x5629c043ef2a]
        2229260:20240615:150124.532 8: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f211ae53520]
        2229260:20240615:150124.532 7: /usr/sbin/zabbix_agentd: collector [processing data](collect_stats_diskdevices+0x40) [0x5629c04224d0]
        2229260:20240615:150124.532 6: /usr/sbin/zabbix_agentd: collector [processing data](zbx_collector_thread+0xc3) [0x5629c0420dd3]
        2229260:20240615:150124.532 5: /usr/sbin/zabbix_agentd: collector [processing data](zbx_thread_start+0x24) [0x5629c043b614]
        2229260:20240615:150124.532 4: /usr/sbin/zabbix_agentd: collector [processing data](MAIN_ZABBIX_ENTRY+0x52d) [0x5629c0403add]
        2229260:20240615:150124.532 3: /usr/sbin/zabbix_agentd: collector [processing data](main+0x33c) [0x5629c04019dc]
        2229260:20240615:150124.532 2: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f211ae3ad90]
        2229260:20240615:150124.532 1: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f211ae3ae40]
        2229260:20240615:150124.532 0: /usr/sbin/zabbix_agentd: collector [processing data](_start+0x25) [0x5629c0401a85]
        2229260:20240615:150124.532 === Memory map: ===
        2229260:20240615:150124.532 5629c03e6000-5629c03ff000 r--p 00000000 fc:02 5317 /usr/sbin/zabbix_agentd
        2229260:20240615:150124.532 5629c03ff000-5629c0475000 r-xp 00019000 fc:02 5317 /usr/sbin/zabbix_agentd
        2229260:20240615:150124.532 5629c0475000-5629c049d000 r--p 0008f000 fc:02 5317 /usr/sbin/zabbix_agentd
        2229260:20240615:150124.532 5629c049d000-5629c049f000 r--p 000b6000 fc:02 5317 /usr/sbin/zabbix_agentd
        2229260:20240615:150124.532 5629c049f000-5629c04a1000 rw-p 000b8000 fc:02 5317 /usr/sbin/zabbix_agentd
        2229260:20240615:150124.532 5629c04a1000-5629c04a2000 rw-p 00000000 00:00 0
        2229260:20240615:150124.532 5629c1484000-5629c14cb000 rw-p 00000000 00:00 0 [heap]
        2229260:20240615:150124.532 7f211a3cf000-7f211a3d2000 r--p 00000000 fc:02 3384 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
        2229260:20240615:150124.532 7f211a3d2000-7f211a3e9000 r-xp 00003000 fc:02 3384 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
        2229260:20240615:150124.532 7f211a3e9000-7f211a3ed000 r--p 0001a000 fc:02 3384 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
        2229260:20240615:150124.532 7f211a3ed000-7f211a3ee000 r--p 0001d000 fc:02 3384 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
        2229260:20240615:150124.532 7f211a3ee000-7f211a3ef000 rw-p 0001e000 fc:02 3384 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
        2229260:20240615:150124.532 7f211a3ef000-7f211a3fc000 rw-s 00000000 00:01 27525170 /SYSV00000000 (deleted)
        2229260:20240615:150124.532 7f211a3fc000-7f211a479000 rw-s 00000000 00:01 27525169 /SYSV00000000 (deleted)
        2229260:20240615:150124.532 7f211a479000-7f211a47f000 rw-p 00000000 00:00 0
        2229260:20240615:150124.532 7f211a47f000-7f211a482000 r--p 00000000 fc:02 8982 /usr/lib/x86_64-linux-gnu/libresolv.so.2
        2229260:20240615:150124.532 7f211a482000-7f211a48c000 r-xp 00003000 fc:02 8982 /usr/lib/x86_64-linux-gnu/libresolv.so.2
        2229260:20240615:150124.532 7f211a48c000-7f211a48f000 r--p 0000d000 fc:02 8982 /usr/lib/x86_64-linux-gnu/libresolv.so.2
        2229260:20240615:150124.532 7f211a48f000-7f211a490000 r--p 0000f000 fc:02 8982 /usr/lib/x86_64-linux-gnu/libresolv.so.2
        2229260:20240615:150124.532 7f211a490000-7f211a491000 rw-p 00010000 fc:02 8982 /usr/lib/x86_64-linux-gnu/libresolv.so.2
        2229260:20240615:150124.532 7f211a491000-7f211a495000 rw-p 00000000 00:00 0
        2229260:20240615:150124.532 7f211a495000-7f211a497000 r--p 00000000 fc:02 8861 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9
        2229260:20240615:150124.532 7f211a497000-7f211a499000 r-xp 00002000 fc:02 8861 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9
        2229260:20240615:150124.532 7f211a499000-7f211a49a000 r--p 00004000 fc:02 8861 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9
        2229260:20240615:150124.532 7f211a49a000-7f211a49b000 r--p 00004000 fc:02 8861 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9
        2229260:20240615:150124.532 7f211a49b000-7f211a49c000 rw-p 00005000 fc:02 8861 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9
        2229260:20240615:150124.532 7f211a49c000-7f211a49e000 r--p 00000000 fc:02 8801 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0
        .....
        2229260:20240615:150124.536 7f211b822000-7f211b823000 rw-s 00000000 00:01 27525168 /SYSV00000000 (deleted)
        2229260:20240615:150124.536 7f211b823000-7f211b825000 r--p 00037000 fc:02 8319 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
        2229260:20240615:150124.536 7f211b825000-7f211b827000 rw-p 00039000 fc:02 8319 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
        2229260:20240615:150124.536 7ffefd9e6000-7ffefda07000 rw-p 00000000 00:00 0 [stack]
        2229260:20240615:150124.536 7ffefdbdc000-7ffefdbe0000 r--p 00000000 00:00 0 [vvar]
        2229260:20240615:150124.536 7ffefdbe0000-7ffefdbe2000 r-xp 00000000 00:00 0 [vdso]
        2229260:20240615:150124.536 ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0 [vsyscall]
        2229260:20240615:150124.536 ================================
        2229260:20240615:150124.536 Please consider attaching a disassembly listing to your bug report.
        2229260:20240615:150124.536 This listing can be produced with, e.g., objdump -DSswx zabbix_agentd.
        2229260:20240615:150124.536 ================================
        2229259:20240615:150124.537 One child process died (PID:2229260,exitcode/signal:1). Exiting ...
        zabbix_agentd [2229259]: Error waiting for process with PID 2229260: [10] No child processes
        2229259:20240615:150124.546 Zabbix Agent stopped. Zabbix 7.0.0 (revision 49955f1fb5c).​
        root@thanos:/home/whanklee# sudo systemctl status zabbix-agent
        Code:
        ● zabbix-agent.service - Zabbix Agent
             Loaded: loaded (/lib/systemd/system/zabbix-agent.service; enabled; vendor preset: enabled)
             Active: activating (auto-restart) (Result: exit-code) since Sat 2024-06-15 14:51:39 CEST; 2s ago
            Process: 2224778 ExecStart=/usr/sbin/zabbix_agentd -c $CONFFILE (code=exited, status=0/SUCCESS)
            Process: 2224835 ExecStop=/bin/sh -c [ -n "$1" ] && kill -s TERM "$1" -- $MAINPID (code=exited, status=1/FAILURE)
           Main PID: 2224780 (code=exited, status=0/SUCCESS)
                CPU: 237ms​

        journalctl -u zabbix-agent
        Code:
        Jun 15 14:52:23 thanos.xxxxxx.xxx systemd[1]: Starting Zabbix Agent...
        Jun 15 14:52:23 thanos.xxxxxx.xxx systemd[1]: Started Zabbix Agent.
        Jun 15 14:52:24 thanos.xxxxxx.xxx systemd[1]: zabbix-agent.service: Control process exited, code=exited, status=1/FAILURE
        Jun 15 14:52:24 thanos.xxxxxx.xxx systemd[1]: zabbix-agent.service: Failed with result 'exit-code'.
        Jun 15 14:52:35 thanos.xxxxxx.xxx systemd[1]: zabbix-agent.service: Scheduled restart job, restart counter is at 2065.
        Jun 15 14:52:35 thanos.xxxxxx.xxx systemd[1]: Stopped Zabbix Agent.​
        Someone told me it can be the problem:

        https://github.com/zabbix/zabbix/pull/125

        Comment

        • Markku
          Senior Member
          Zabbix Certified SpecialistZabbix Certified ProfessionalZabbix Certified Expert
          • Sep 2018
          • 1781

          #5
          To verify if the agent crash reason is the specific passive item, you can capture the TCP 10050 traffic on the agent (or on the server, filtering with the agent IP) and see at which request it specifically crashes.

          Since you are using TLS, you need either to arrange for the TLS decryption in Wireshark (see https://majornetwork.net/2023/10/dec...ith-wireshark/) or just disable TLS for a moment to capture cleartext traffic.

          (Edit: Or, increase the agent logging so that it logs the received requests, that logging should occur before the crash.)

          Markku

          Comment

          • Markku
            Senior Member
            Zabbix Certified SpecialistZabbix Certified ProfessionalZabbix Certified Expert
            • Sep 2018
            • 1781

            #6
            And now that I think about this, it probably isn't just passive item that could trigger the crash, active item should be able to do that as well, but in any case increasing the agent logging should reveal the crash item.

            Markku

            Comment

            • Markku
              Senior Member
              Zabbix Certified SpecialistZabbix Certified ProfessionalZabbix Certified Expert
              • Sep 2018
              • 1781

              #7
              Yeah, both vfs.dev.read and vfs.dev.write crash here as well as active items. ZBX-24455 is being solved I see.

              Markku

              Comment

              • whanklee
                Junior Member
                • Dec 2019
                • 19

                #8
                Okay. What shoud I do now? How can I fix it?

                Comment

                • Markku
                  Senior Member
                  Zabbix Certified SpecialistZabbix Certified ProfessionalZabbix Certified Expert
                  • Sep 2018
                  • 1781

                  #9
                  Originally posted by whanklee
                  Okay. What shoud I do now? How can I fix it?
                  Go back to some previous agent version while waiting for a fixed 7.0.x agent, or disable the failing items.

                  Markku

                  Comment

                  • whanklee
                    Junior Member
                    • Dec 2019
                    • 19

                    #10
                    I will reinstall Zabbix Agent 6.4.x and try it.

                    Comment

                    • whanklee
                      Junior Member
                      • Dec 2019
                      • 19

                      #11
                      I can confirme that Zabbix Agent 6.4.1 works well. There are sose bugs in Zabbix Agent 7.0.x
                      My problem is solved.

                      Comment

                      Working...