Ad Widget

Collapse

1000 VPS proxy always with high queue

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • jjeff123
    Member
    • May 2022
    • 33

    #1

    1000 VPS proxy always with high queue

    Server and proxy = 7.0.12, but this has been an issue for every version of 7 I've used.
    One proxy always has a queue of 60-100K values but I see no reason why.

    We have a couple dozen proxies, almost exclusively monitoring network gear via SNMP queries, no traps.
    It's also the largest proxy, with 1000 VPS.
    But the 2nd place proxy is still 680 VPS, and has a queue of zero. The entire server queue is from this one problem proxy.
    But looking at every stat I can think of, I just don't see anything that should limit us.
    Using a fraction of CPU/memory/disk IO on the proxy.
    Server recently upgraded and has no issues.

    I know we're limited to 1000 items per second as the MaxConcurrentChecksPerPoller, but I've forked plenty of pollers, probably too many, but nothing seems to help.
    Unless 1000 is a hard limit someplace else on a proxy?

    I also know we're using the old version 6 type sync SNMP templates, not the newer v 7 async templates. That's the only problem I know of.

    Can I fix my problem by throwing money at it and putting a 2nd proxy at this site and a proxy group?

    Proxy settings:

    CacheSize=256M
    HistoryCacheSize=64M
    HistoryIndexCacheSize=32M
    ProxyMemoryBufferSize=128M
    StartVMwareCollectors=1
    # we don't use any vmware at this facility, but I have this as standard on all our proxies
    VMwareFrequency=60
    VMwarePerfFrequency=60
    VMwareCacheSize=16M
    ProxyOfflineBuffer=48
    StartDiscoverers=3
    StartPollers=20
    StartSNMPPollers=20
    StartPingers=6
    StartPreprocessors=6
    StartDBSyncers=6
    Timeout=3
    StartPollersUnreachable=3
    MaxConcurrentChecksPerPoller=1000

    Graphs below show some atypical activity levels in the last 30 minutes as I've been monkeying with settings (again) to improve performance.




    ​​

    ​​
    ​​
  • jjeff123
    Member
    • May 2022
    • 33

    #2
    Possibly answered my own question. Maybe, I don't know.

    There were 3 things going on with this proxy.
    - It had 2 IP addresses bound to the NIC (somehow had both my assigned static IP, and the original DHCP IP from when we deployed it)
    - 100+ days uptime
    - I did an apt-get update, which included a kernel update, followed by a reboot

    So beats me. I guess if you're reading this, apt-get update and reboot.

    Comment

    • salavie
      Junior Member
      • Oct 2020
      • 29

      #3
      Check if the proxy and server time are consistent

      Comment

      • jjeff123
        Member
        • May 2022
        • 33

        #4
        Originally posted by salavie
        Check if the proxy and server time are consistent
        See my prior post.
        All fixed, something goofy.

        Comment

        Working...