Ad Widget

Collapse

Apache by Zabbix Agent issues

Collapse
This topic has been answered.
X
X
 
  • Time
  • Show
Clear All
new posts
  • sean@bigblockstudios.ca
    Junior Member
    • Jul 2023
    • 4

    #1

    Apache by Zabbix Agent issues

    Hi,

    I've been struggling for several hours with setting up the apache template on one particular host, and now it's time to start begging for help

    - other services on the host have been monitored for a couple of years now, networking/dns/communication with the Zabbix server is not the issue.
    - I have verified the availability of the server-status page from the Zabbix server (on a different host in same network) and the agent the apache template is installed on
    - What appears to be identical apache monitoring on 2 other hosts is running just fine. - no issues

    This particular host is not tracking the apache data (no graphs) checking the logs on both hosts it appears that "Preprocessing" is failing even though the data is there:
    Looking at just the BusyWorkers (all the variables fail)


    Please take a look at the log items below, any help would be much appraciated!!




    Code:
    1641527:20230728:133035.737 error reason for "www.my_fake_server_name.com:apache.workers_total.busy" changed: Preprocessing failed for: {"Server":"nginx","Date":"Fri, 28 Jul 2023 17:30:35 GMT","Type":"text/html; charset=ISO-8859-1","...
    1. Failed: cannot extract value from json by path "$.BusyWorkers": no data matches the specified path​
    Honestly don't know if this is the same host, I can't find any real indication
    Code:
    1641517:20230728:133135.298 worker_preprocess_dep_items: FAIL Preprocessing failed for: {"Server":"nginx","Date":"Fri, 28 Jul 2023 17:31:35 GMT","Type":"text/html; charset=ISO-8859-1","...
    1. Failed: cannot extract value from json by path "$.BusyWorkers": no data matches the specified path
    1641517:20230728:133135.299 worker_preprocess_dep_items(): {"Server":"nginx","Date":"Fri, 28 Jul 2023 17:31:35 GMT","Type":"text/html; charset=ISO-8859-1","Length":1995,"Connection":"keep-alive","Vary":"Accept-Encoding","Encoding":"gzip","Server Version":"Apache/2.4.41 (Ubuntu) OpenSSL/1.1.1f mod_fcgid/2.3.9</dt>","Server MPM":"event</dt>","Server Built":"2023-03-08T17:32:54","Current Time":"Friday, 28-Jul-2023 17:31:35 UTC</dt>","Restart Time":"Friday, 28-Jul-2023 17:30:02 UTC</dt>"," Generation":"1</dt>","Parent Server MPM Generation":"0</dt>","Server uptime":" 1 minute 33 seconds</dt>","Server load":"0.17 0.20 0.14</dt>","Total accesses":"4 - Total Traffic: 21 kB - Total Duration: 3808</dt>","CPU Usage":"u.6 s.11 cu0 cs0 - .763% CPU load</dt>","Total FastCGI processes":1,"Process":"cgi_wrapper</b>&nbsp;&nbsp;(/var/www/cgi-bin/cgi_wrapper/cgi_wrapper)<br />","cache type":"<b>SHMCB</b>, shared memory: <b>512000</b> bytes, current entries: <b>5</b><br>subcaches: <b>32</b>, indexes per subcache: <b>88</b><br>time left on oldest entries' objects: avg: <b>249</b> seconds, (range: 249...249)<br>index usage: <b>0%</b>, cache usage: <b>0%</b><br>total entries stored since starting: <b>5</b><br>total entries replaced since starting: <b>0</b><br>total entries expired since starting: <b>0</b><br>total (pre-expiry) entries scrolled out of the cache: <b>0</b><br>total retrieves since starting: <b>0</b> hit, <b>0</b> miss<br>total removes since starting: <b>0</b> hit, <b>0</b> miss<br></td></tr>","ServerVersion":"nginx","Workers":{"waiting":0,"starting":0,"reading":0,"sending":0,"keepalive":0,"dnslookup":0,"closing":0,"logging":0,"finishing":0,"cleanup":0,"slot":0}}​
    Code:
    root@monitor:/etc/zabbix# curl -get http://my_fake_server_ip/server-status-95mf873hgy0?auto
    my_fake_server_ip
    ServerVersion: Apache/2.4.41 (Ubuntu) OpenSSL/1.1.1f mod_fcgid/2.3.9
    ServerMPM: event
    Server Built: 2023-03-08T17:32:54
    CurrentTime: Friday, 28-Jul-2023 17:32:23 UTC
    RestartTime: Friday, 28-Jul-2023 17:30:02 UTC
    ParentServerConfigGeneration: 1
    ParentServerMPMGeneration: 0
    ServerUptimeSeconds: 141
    ServerUptime: 2 minutes 21 seconds
    Load1: 0.23
    Load5: 0.22
    Load15: 0.15
    Total Accesses: 35
    Total kBytes: 111
    Total Duration: 8447
    CPUUser: .81
    CPUSystem: .17
    CPUChildrenUser: 0
    CPUChildrenSystem: 0
    CPULoad: .695035
    Uptime: 141
    ReqPerSec: .248227
    BytesPerSec: 806.128
    BytesPerReq: 3247.54
    DurationPerReq: 241.343
    BusyWorkers: 1
    IdleWorkers: 49
    Processes: 2
    Stopping: 0
    BusyWorkers: 1
    IdleWorkers: 49
    ConnsTotal: 0
    ConnsAsyncWriting: 0
    ConnsAsyncKeepAlive: 0
    ConnsAsyncClosing: 0
    Scoreboard: W_________________________________________________....................................................................................................
    TLSSessionCacheStatus
    CacheType: SHMCB
    CacheSharedMemory: 512000
    CacheCurrentEntries: 32
    CacheSubcaches: 32
    CacheIndexesPerSubcaches: 88
    CacheTimeLeftOldestAvg: 247
    CacheTimeLeftOldestMin: 201
    CacheTimeLeftOldestMax: 259
    CacheIndexUsage: 1%
    CacheUsage: 1%
    CacheStoreCount: 32
    CacheReplaceCount: 0
    CacheExpireCount: 0
    CacheDiscardCount: 0
    CacheRetrieveHitCount: 0
    CacheRetrieveMissCount: 0
    CacheRemoveHitCount: 0
    CacheRemoveMissCount: 0​
  • Answer selected by [email protected] at 31-07-2023, 16:58.
    sean@bigblockstudios.ca
    Junior Member
    • Jul 2023
    • 4

    Originally posted by Hamardaban
    Is it working? That's good!
    The question is closed.
    P.S.
    In order to understand what I wrote about, read all the documentation
    and then understand to the end how data is collected and processed in zabbix.​
    Solid advice ...

    Question is resolved without actually finding the cause/issue It seems the data collection was delayed by several days for an undetermined reason.

    Comment

    • Hamardaban
      Senior Member
      Zabbix Certified SpecialistZabbix Certified Professional
      • May 2019
      • 2713

      #2
      In the first two log listings, you have a response from the nginx server, not apache.
      The response format is different, which is why the json search does not work.

      Comment

      • sean@bigblockstudios.ca
        Junior Member
        • Jul 2023
        • 4

        #3
        Originally posted by Hamardaban
        In the first two log listings, you have a response from the nginx server, not apache.
        The response format is different, which is why the json search does not work.
        Thanks Hamardaban

        nginx is acting as a proxy on both servers: this one works (collects data) & shows it as nginix:

        Code:
        [root@manage]# curl -v http://000.000.000.000/server-status-95mf873h32b8g?auto
        * About to connect() to 000.000.000.000 port 80 (#0)
        *   Trying 000.000.000.000...
        * Connected to 000.000.000.000 (000.000.000.000) port 80 (#0)
        > GET /server-status-95mf873vf?auto HTTP/1.1
        > User-Agent: curl/7.29.0
        > Host: 000.000.000.000
        > Accept: */*
        >
        < HTTP/1.1 200 OK
        < Server: nginx
        < Date: Mon, 31 Jul 2023 13:43:13 GMT
        < Content-Type: text/plain; charset=ISO-8859-1
        < Content-Length: 475
        < Connection: keep-alive
        <
        Total Accesses: 3977003
        Total kBytes: 57719342
        CPULoad: .0039145
        Uptime: 1787969
        ReqPerSec: 2.22431
        BytesPerSec: 33056.8
        BytesPerReq: 14861.6
        BusyWorkers: 1
        IdleWorkers: 59
        ConnsTotal: 0
        ConnsAsyncWriting: 0
        ConnsAsyncKeepAlive: 0
        ConnsAsyncClosing: 0
        Scoreboard: ______________W_____________________________________________......................................................................................................................................................
        * Connection #0 to host 000.000.000.000 left intact​

        This is the server that is not tracking data....

        Code:
        root@ip-111-111-111-111:/home/ubuntu# curl -v 111.111.111.111/server-status-95mf873hgy0?auto
        *   Trying 111.111.111.111:80...
        * TCP_NODELAY set
        * Connected to 111.111.111.111 (111.111.111.111) port 80 (#0)
        > GET /server-status-95mf873hgy0?auto HTTP/1.1
        > Host: 111.111.111.111
        > User-Agent: curl/7.68.0
        > Accept: */*
        >
        * Mark bundle as not supporting multiuse
        < HTTP/1.1 200 OK
        < Server: nginx
        < Date: Mon, 31 Jul 2023 13:39:43 GMT
        < Content-Type: text/plain; charset=ISO-8859-1
        < Content-Length: 1433
        < Connection: keep-alive
        < Vary: Accept-Encoding
        <
        111.111.111.111
        ServerVersion: Apache/2.4.41 (Ubuntu) OpenSSL/1.1.1f mod_fcgid/2.3.9
        ServerMPM: event
        Server Built: 2023-03-08T17:32:54
        CurrentTime: Monday, 31-Jul-2023 13:39:43 UTC
        RestartTime: Friday, 28-Jul-2023 17:30:02 UTC
        ParentServerConfigGeneration: 7
        ParentServerMPMGeneration: 6
        ServerUptimeSeconds: 245381
        ServerUptime: 2 days 20 hours 9 minutes 41 seconds
        Load1: 0.13
        Load5: 0.13
        Load15: 0.10
        Total Accesses: 54281
        Total kBytes: 617023
        Total Duration: 31611823
        CPUUser: 70
        CPUSystem: 28.06
        CPUChildrenUser: 18967.8
        CPUChildrenSystem: 927.38
        CPULoad: 8.14782
        Uptime: 245381
        ReqPerSec: .221211
        BytesPerSec: 2574.9
        BytesPerReq: 11640
        DurationPerReq: 582.374
        BusyWorkers: 1
        IdleWorkers: 49
        Processes: 2
        Stopping: 0
        BusyWorkers: 1
        IdleWorkers: 49
        ConnsTotal: 0
        ConnsAsyncWriting: 0
        ConnsAsyncKeepAlive: 0
        ConnsAsyncClosing: 0
        Scoreboard: ___________________W______________________________....................................................................................................
        TLSSessionCacheStatus
        CacheType: SHMCB
        CacheSharedMemory: 512000
        CacheCurrentEntries: 96
        CacheSubcaches: 32
        CacheIndexesPerSubcaches: 88
        CacheTimeLeftOldestAvg: 138
        CacheTimeLeftOldestMin: 0
        CacheTimeLeftOldestMax: 298
        CacheIndexUsage: 3%
        CacheUsage: 4%
        CacheStoreCount: 9251
        CacheReplaceCount: 0
        CacheExpireCount: 9115
        CacheDiscardCount: 0
        CacheRetrieveHitCount: 0
        CacheRetrieveMissCount: 0
        CacheRemoveHitCount: 40
        CacheRemoveMissCount: 0​

        So that is confusing, server 000.000.000.000 is collecting data, but it doesn't look like all of it, while server 111.111.111.111 does not collect data, but the response appears to be correct (even though there is much more info there). Both servers are using event MPM.

        What am I missing here?

        Comment

        • Hamardaban
          Senior Member
          Zabbix Certified SpecialistZabbix Certified Professional
          • May 2019
          • 2713

          #4
          The problem is that dependent items cannot find the desired json path in the listing of the main (master,raw) item.
          I think that the fact that you use nginx as a proxy damage the JSON information received by zabbix.

          Change the settings of the master item (which makes a request to the web server) so that it saves the data.
          Wait for the configuration update or update it yourself.
          After that, in the "latest data" section, find the data for this item and post here. Do this for both servers.
          Last edited by Hamardaban; 31-07-2023, 16:07.

          Comment

          • sean@bigblockstudios.ca
            Junior Member
            • Jul 2023
            • 4

            #5
            Not sure I follow you .... but it appears zabbix has started tracking the apache data! ?? - about an hour ago, I made no configuration changes, just executed those manual curl requests.
            Attached Files

            Comment

            • Hamardaban
              Senior Member
              Zabbix Certified SpecialistZabbix Certified Professional
              • May 2019
              • 2713

              #6
              Is it working? That's good!
              The question is closed.
              P.S.
              In order to understand what I wrote about, read all the documentation
              and then understand to the end how data is collected and processed in zabbix.​

              Comment

              • sean@bigblockstudios.ca
                Junior Member
                • Jul 2023
                • 4

                #7
                Originally posted by Hamardaban
                Is it working? That's good!
                The question is closed.
                P.S.
                In order to understand what I wrote about, read all the documentation
                and then understand to the end how data is collected and processed in zabbix.​
                Solid advice ...

                Question is resolved without actually finding the cause/issue It seems the data collection was delayed by several days for an undetermined reason.

                Comment

                Working...