Template OS Linux SSH - Errors during ssh access

[hpesata]

Hi!

We are using Zabbix 5.4.0 to monitor our servers.

We have 2 Linux servers where the monitoring using ssh access doesnt work.

It runs well for some time and suddenly the ssh connection cant be established anymore, after a while it works again.

We see this in the logs:

zabbix_server.log

67783:20210727:192435.161 item "myserver:ssh.run[system.cpu.util.idle,{HOST.CONN},{$PORT}]" became not supported: Cannot establish SSH session: Socket error: Connection reset by peer
67783:20210727:192436.167 item "myserver:ssh.run[system.cpu.switches,{HOST.CONN},{$PORT}]" became not supported: Cannot establish SSH session: Socket error: Connection reset by peer
67783:20210727:192437.182 item "myserver:ssh.run[system.cpu.intr,{HOST.CONN},{$PORT}]" became not supported: Cannot establish SSH session: Socket error: Connection reset by peer
67783:20210727:192438.199 item "myserver:ssh.run[system.boottime,{HOST.CONN},{$PORT}]" became not supported: Cannot establish SSH session: Socket error: Connection reset by peer
67783:20210727:192439.214 item "myserver:ssh.run[sshkey.mem_available,{HOST.CONN},{$PORT}]" became not supported: Cannot establish SSH session: Socket error: Connection reset by peer
67783:20210727:192440.231 item "myserver:ssh.run[proc.num.run,{HOST.CONN},{$PORT}]" became not supported: Cannot establish SSH session: Socket error: Connection reset by peer
67782:20210727:192919.218 item "myserver:ssh.run[system_cpu_load_percpu_avg1,{HOST.CONN},{$PORT}]" became supported
67784:20210727:192928.243 item "myserver:ssh.run[system.cpu.util.user,{HOST.CONN},{$PORT}]" became supported
67783:20210727:192929.245 item "myserver:ssh.run[system.cpu.util.steal,{HOST.CONN},{$PORT}]" became supported
67785:20210727:192931.246 item "myserver:ssh.run[system.cpu.util.softirq,{HOST.CONN},{$PORT}]" became supported
67782:20210727:192932.251 item "myserver:ssh.run[system.cpu.util.nice,{HOST.CONN},{$PORT}]" became supported
67783:20210727:192933.251 item "myserver:ssh.run[system.cpu.util.iowait,{HOST.CONN},{$PORT}]" became supported
67785:20210727:192934.253 item "myserver:ssh.run[system.cpu.util.interrupt,{HOST.CONN},{$PORT}]" became supported
67785:20210727:192935.260 item "myserver:ssh.run[system.cpu.util.idle,{HOST.CONN},{$PORT}]" became supported
67784:20210727:192938.264 item "myserver:ssh.run[system.boottime,{HOST.CONN},{$PORT}]" became supported
67783:20210727:192940.269 item "myserver:ssh.run[proc.num.run,{HOST.CONN},{$PORT}]" became supported

On myserver:

/var/log/secure

Jul 27 19:35:12 myserver sshd[14776]: pam_unix(sshd:session): session closed for user zabbix
Jul 27 19:35:12 myserver sshd[14796]: Accepted publickey for zabbix from 10.49.166.163 port 41328 ssh2
Jul 27 19:35:12 myserver sshd[14796]: pam_unix(sshd:session): session opened for user zabbix by (uid=0)
Jul 27 19:35:12 myserver sshd[14796]: pam_unix(sshd:session): session closed for user zabbix
Jul 27 19:35:13 myserver sshd[14816]: Accepted publickey for zabbix from 10.49.166.163 port 41334 ssh2
Jul 27 19:35:13 myserver sshd[14816]: pam_unix(sshd:session): session opened for user zabbix by (uid=0)
Jul 27 19:35:13 myserver sshd([14816]: pam_unix(sshd:session): session closed for user zabbix
Jul 27 19:35:14 myserver sshd[14836]: Accepted publickey for zabbix from 10.49.166.163 port 41340 ssh2
Jul 27 19:35:14 myserver sshd[14836]: pam_unix(sshd:session): session opened for user zabbix by (uid=0)
Jul 27 19:35:14 myserver sshd[14836]: pam_unix(sshd:session): session closed for user zabbix
Jul 27 19:35:15 myserver sshd[14857]: fatal: Write failed: Connection reset by peer
Jul 27 19:35:16 myserver sshd[14860]: fatal: Write failed: Connection reset by peer
Jul 27 19:35:18 myserver sshd[14865]: fatal: Write failed: Connection reset by peer
Jul 27 19:35:19 myserver sshd[14867]: fatal: Write failed: Connection reset by peer
Jul 27 19:35:20 myserver sshd[14869]: fatal: Write failed: Connection reset by peer
Jul 27 19:35:33 myserver sshd[14873]: fatal: Write failed: Connection reset by peer
Jul 27 19:35:34 myserver sshd[15367]: Did not receive identification string from 10.49.166.163

We tried several things with no success so far.

Any hints on how to find out what is going on, would be greatly appreciated!

Thanx in advance,
Hans