Agent startup Failure between Sparc T4-1 and Sparc T4-2

[mrmurdock]

I finally have zabbix 4.4.8 agent compiled on Sparc T4-1 and Sparc T4-2, under solaris 11.4. On the T4-1 and the T4-2 the agent compiled. The T4-1 is running a latest OS updates (11.4-11.4.20.0.1.4.0), vs the T4-2 (11.4-11.4.3.0.1.5.0).
<code>
./configure --enable-agent --with-libpcre-include=/usr/include/pcre (the same on both boxes).
</code>
However, the agent starts on the T4-1 box, with no issues. On the T4-2 box I get the following
<code>
11832:20200507:115435.679 Starting Zabbix Agent [Zabbix server]. Zabbix 4.4.8 (revision 3d7663fd7c).
11832:20200507:115435.680 **** Enabled features ****
11832:20200507:115435.680 IPv6 support: NO
11832:20200507:115435.680 TLS support: NO
11832:20200507:115435.680 **************************
11832:20200507:115435.680 using configuration file: /usr/local/etc/zabbix_agentd.conf
11832:20200507:115435.684 agent #0 started [main process]
11833:20200507:115435.685 agent #1 started [collector]
11834:20200507:115435.686 agent #2 started[listener #1]
11834:20200507:115435.687 Got signal [signal:11(SIGSEGV),reason:1,refaddr:7f5c321c]. Crashing ...
11834:20200507:115435.687 ====== Fatal information: ======
11834:20200507:115435.687 program counter not available for this architecture
11834:20200507:115435.687 === Registers: ===
11835:20200507:115435.687 agent #3 started[listener #2]
11834:20200507:115435.688 register dump not available for this architecture
11834:20200507:115435.688 === Backtrace: ===
11835:20200507:115435.688 Got signal [signal:11(SIGSEGV),reason:1,refaddr:7f5c321c]. Crashing ...
11835:20200507:115435.688 ====== Fatal information: ======
11835:20200507:115435.688 program counter not available for this architecture
11835:20200507:115435.688 === Registers: ===
11835:20200507:115435.688 register dump not available for this architecture
11834:20200507:115435.688 11: /usr/local/sbin/zabbix_agentd'zbx_backtrace+0x3c [0x100041854]
11835:20200507:115435.689 === Backtrace: ===
11834:20200507:115435.689 10: /usr/local/sbin/zabbix_agentd'zbx_log_fatal_info+0x208 [0x100041bac]
11834:20200507:115435.689 9: /usr/local/sbin/zabbix_agentd'fatal_signal_handler+0x18 [0x100041f38]
11834:20200507:115435.689 8: /lib/sparcv9/libc.so.1'__sighndlr+0xc [0xffffffff7eae1898]
11834:20200507:115435.689 7: /lib/sparcv9/libc.so.1'call_user_handler+0x354 [0xffffffff7ead41ac]
11834:20200507:115435.689 6: /lib/sparcv9/libc.so.1'sigacthandler+0x54 [0xffffffff7ead44ec]
11834:20200507:115435.689 5: /usr/local/sbin/zabbix_agentd'listener_thread+0x1bc [0x1000240cc]
11834:20200507:115435.689 4: /usr/local/sbin/zabbix_agentd'zbx_thread_start+0x1c [0x100040270]
11834:20200507:115435.689 3: /usr/local/sbin/zabbix_agentd'MAIN_ZABBIX_ENTRY+0x4f8 [0x10001d934]
11834:20200507:115435.689 2: /usr/local/sbin/zabbix_agentd'daemon_start+0x224 [0x10004121c]
11834:20200507:115435.689 1: /usr/local/sbin/zabbix_agentd'main+0x170 [0x10001dc38]
11834:20200507:115435.689 0: /usr/local/sbin/zabbix_agentd'_start+0x64 [0x10001b684]
11834:20200507:115435.689 === Memory map: ===
11834:20200507:115435.689 memory map not available for this platform
11835:20200507:115435.690 11: /usr/local/sbin/zabbix_agentd'zbx_backtrace+0x3c [0x100041854]
11835:20200507:115435.690 10: /usr/local/sbin/zabbix_agentd'zbx_log_fatal_info+0x208 [0x100041bac]
11835:20200507:115435.690 9: /usr/local/sbin/zabbix_agentd'fatal_signal_handler+0x18 [0x100041f38]
11835:20200507:115435.690 8: /lib/sparcv9/libc.so.1'__sighndlr+0xc [0xffffffff7eae1898]
11834:20200507:115435.690 ================================
11835:20200507:115435.690 7: /lib/sparcv9/libc.so.1'call_user_handler+0x354 [0xffffffff7ead41ac]
11835:20200507:115435.690 6: /lib/sparcv9/libc.so.1'sigacthandler+0x54 [0xffffffff7ead44ec]
11835:20200507:115435.690 5: /usr/local/sbin/zabbix_agentd'listener_thread+0x1bc [0x1000240cc]
11835:20200507:115435.690 4: /usr/local/sbin/zabbix_agentd'zbx_thread_start+0x1c [0x100040270]
11835:20200507:115435.690 3: /usr/local/sbin/zabbix_agentd'MAIN_ZABBIX_ENTRY+0x4f8 [0x10001d934]
11835:20200507:115435.690 2: /usr/local/sbin/zabbix_agentd'daemon_start+0x224 [0x10004121c]
11835:20200507:115435.690 1: /usr/local/sbin/zabbix_agentd'main+0x170 [0x10001dc38]
11835:20200507:115435.690 0: /usr/local/sbin/zabbix_agentd'_start+0x64 [0x10001b684]
11836:20200507:115435.691 agent #4 started[listener #3]
11835:20200507:115435.691 === Memory map: ===
11835:20200507:115435.691 memory map not available for this platform
11835:20200507:115435.691 ================================
11836:20200507:115435.691 Got signal [signal:11(SIGSEGV),reason:1,refaddr:7f5c321c]. Crashing ...
11836:20200507:115435.691 ====== Fatal information: ======
11836:20200507:115435.691 program counter not available for this architecture
11832:20200507:115435.691 One child process died (PID:11834,exitcode/signal:1). Exiting ...
11836:20200507:115435.692 === Registers: ===
11832:20200507:115435.695 cannot remove shared memory for collector: [22] Invalid argument
11832:20200507:115435.695 Zabbix Agent stopped. Zabbix 4.4.8 (revision 3d7663fd7c).
</code>

[tim.mooney]

Did you compile it on just one host and then try use it on both? Or did you compile each agent separately on the separate hosts?

gcc or the SunPro compiler? Are you comfortable using a debugger like gdb or dbx to examine the call stack and function arguments at the point of the crash?

[mrmurdock]

Compiled it separately on both hosts. I just finished compiling on its Twin T4-2 with the same result. I did compile and run on a T5-2 (after I posted the first thread) with no issues.
I used gcc that Oracle Solaris provides.
really not comfortable, but not shy to try.
This is crazy, if it had a problem, why would it have configured and compiled (make install) and not found a issue during those times? It has to check to make sure the platform is supported and it can test read and write memory segments and such, right,? at least I would think so.
I just install gdb on the host.

[mrmurdock]

Now I just got the same error on a T5120 Platform under Solaris 11.3.

[mrmurdock]

Well, I found a google search that said to disable the compiler optimization with CFLAGS="";./configure --enable-agnet --with-libpcre-include=/usr/include/pcre. IT works that way and I am happy with it.
Thank you.

[tim.mooney]

This is crazy, if it had a problem, why would it have configured and compiled (make install) and not found a issue during those times? It has to check to make sure the platform is supported and it can test read and write memory segments and such, right,? at least I would think so.

No, it's not unusual for even software that completes its feature test and selection via the "configure" script to have compilation failures or, as in your case, to build but not work. Software portability between different Unix and Unix-like platforms can be tricky, especially for stuff that goes low-level. There are lots of subtle differences between different open systems, and even within the same OS (like Solaris), different releases can change things that cause these types of issues. Do a web image search for "Unix family tree" to see how complicated the derivations can become.

Glad that compiling without optimization fixed it for you. That's potentially a bad sign for the gcc compiler you have, but that's a different issue.