Ad Widget

Collapse

http pollers exhaust all memory available

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • cvhelp
    Junior Member
    • Jan 2017
    • 5

    #1

    http pollers exhaust all memory available

    Hello all,
    I've started on new installation Zabbix Server 3.0.10 under AWS Linux to supersede the existing 2.4.
    Instead of upgrading, I've decided to start fresh.

    The issue I'm experiencing is that http pollers memory usage grows over 1 or 2 days until it exhaust all the memory available in the VM and zabbix_server dies.

    Since this issue started, I've added more RAM to the VM: 1GB->2GB and again from 2GB->4GB yesterday.
    With 4GB it seems to be taking more time but looking at the graphs it seems likely that will happen (see Memory used by http pollers including data, stack, virtual and resident size; Overall Memory usage and htop under http://imgur.com/a/a7BHi).
    Could it be a memory leak ?

    I use a template for web scenarios that are applied to a group of hosts. Every host that is monitored, has user macros with the URL that are used by the web scenario. In total, it monitors 90 URLs.
    The web scenario template has the follow configuration:
    • Scenario->Update interval (in sec): 120
    • Scenario->Attempts: 3
    • Steps->Timeout: 15 sec
    • Steps->Status codes: 200
    • Authentication->SSL verify peer: Yes
    • Authentication->SSL verify host: Yes


    Can anyone help to troubleshoot ?
    Thank you.
    Last edited by cvhelp; 25-08-2017, 12:02. Reason: mention htop
  • cvhelp
    Junior Member
    • Jan 2017
    • 5

    #2
    I've been reading maps, smaps and Memory Stats! by James Hunt and tried his diag on one of the http pollers (although the output looks very similar to other http pollers):

    Code:
    $ sudo ./diag 2674
    [heap]:
      private        -   [clean]     194.0 M [dirty]
       shared        -   [clean]     132.0 k [dirty]
    
    [mmap]:
      private        -   [clean]     124.0 k [dirty]
       shared    988.0 k [clean]     404.0 k [dirty]
    
    [stack]:
      private        -   [clean]      96.0 k [dirty]
       shared        -   [clean]         -   [dirty]
    
    [vvar]:
      private        -   [clean]         -   [dirty]
       shared        -   [clean]         -   [dirty]
    And a few minutes later, heap private has increased a few MBs:
    Code:
    $ sudo ./diag 2674
    [heap]:
      private        -   [clean]     197.7 M [dirty]
       shared        -   [clean]     132.0 k [dirty]
    
    [mmap]:
      private        -   [clean]     124.0 k [dirty]
       shared    988.0 k [clean]     404.0 k [dirty]
    
    [stack]:
      private        -   [clean]      96.0 k [dirty]
       shared        -   [clean]         -   [dirty]
    
    [vvar]:
      private        -   [clean]         -   [dirty]
       shared        -   [clean]         -   [dirty]
    Last edited by cvhelp; 25-08-2017, 12:56. Reason: another example

    Comment

    • cvhelp
      Junior Member
      • Jan 2017
      • 5

      #3
      I've ran memleax to the same process above and something weird is happening.
      I've attached the full report.
      Attached Files

      Comment

      Working...