Ad Widget

Collapse

Showing how much data a process is sending/receiving

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • nelsonab
    Senior Member
    Zabbix Certified SpecialistZabbix Certified Professional
    • Sep 2006
    • 1233

    #1

    Showing how much data a process is sending/receiving

    A few people have asked if there is a way to show weather or not you can show how much a process is talking on the network. By default Linux does not have any process accounting for data like this. However SystemTap is one way to accomplish this.

    Here is a sample of the output from a SystemTap script I wrote for this:
    Code:
    COMMAND         DEV     XMIT_PK RECV_PK XMIT_KB RECV_KB
    swapper         eth1       2425    5644     138    6702
    firefox         eth1       2847    3897     354    4501
    X               eth1         77     162       4     191
    zabbix_agentd   lo           16      16       1       0
    zabbix_server   lo            8       8       0       0
    lxpanel         eth1         12      17       0      18
    scsi_eh_1       eth1         10      25       0      32
    ata/0           eth1         10      24       0      31
    vmtoolsd        eth1          2      19       0      23
    gconfd-2        eth1         10      11       0      12
    vmware-user-loa eth1          3       5       0       3
    swapper         eth2          0       5       0       0
    zabbix_agentd   eth1          3       5       0       4
    vmmemctl        eth1          0       7       0       9
    udisks-daemon   eth1          1       2       0       2
    firefox         eth2          0       1       0       0
    lxterminal      eth1          0       1       0       1
    hald-addon-stor eth1          1       1       0       1
    kswapd0         eth1          1       1       0       0
    openbox         eth1          0       1       0       0
    zabbix_server   eth1          0       1       0       1
    SystemTap is a project started by Red Hat to allow for low level instrumentation of various aspects of a running Linux system. Presently it is available for most Linux Distributions. SystemTap scripts need to be compiled into a kernel module before they can be used. On RHEL based systems you will need the systemtap, kernel-debuginfo, kernel-devel and gcc packages to compile the kernel module. However once compiled this module can be taken to another machine with the same kernel revision. All this system will need is the systemtap-runtime package installed.

    To compile and run the module:
    Code:
    # stap -m zabbix zabbix.stp
    The script itself (saved as zabbix.stp):
    Code:
    #SystemTap script for Zabbix
    #*NOTE* You, the end user of this script, are responsible for any risks
    #associated with using this script.
    #Compile string:
    # stap -m zabbix zabbix.stp
    #Information can be found in /proc/systemtap/zabbix/*
    #Current process statistics are found in network
    #clear_on_read sets weather or not the statistics are reset when read
    # the default is 0.
    #The network file is limited to 4096 bytes in size
    
    global ifxmit, ifrecv
    global ifmerged
    global clear_on_read
    
    probe begin 
    {
      clear_on_read=0
    }
    
    probe netdev.transmit
    {
      ifxmit[dev_name, execname()] <<< length
    }
    
    probe netdev.receive
    {
      ifrecv[dev_name, execname()] <<< length
    }
    
    probe procfs("clear_on_read").read
    {
      $value=sprintf("%u",clear_on_read)
    }
    
    probe procfs("clear_on_read").write
    {
      clear_on_read=strtol($value,2)
    }
    
    probe procfs("network").umask(0444).read.maxsize(4096)
    {
      $value=sprintf("%-15s %-7s %7s %7s %7s %7s\n",
             "COMMAND", "DEV", "XMIT_PK", "RECV_PK",
             "XMIT_KB", "RECV_KB")
    
      foreach ([dev, exec] in ifrecv) {
          ifmerged[dev, exec] += @count(ifrecv[dev,exec]);
      }
      foreach ([dev, exec] in ifxmit) {
          ifmerged[dev, exec] += @count(ifxmit[dev,exec]);
      }
      foreach ([dev, exec] in ifmerged-) {
        n_xmit = @count(ifxmit[dev, exec])
        n_recv = @count(ifrecv[dev, exec])
        $value.=sprintf("%-15s %-7s %7d %7d %7d %7d\n",
               exec, dev, n_xmit, n_recv,
               n_xmit ? @sum(ifxmit[dev, exec])/1024 : 0,
               n_recv ? @sum(ifrecv[dev, exec])/1024 : 0)
      }
    
      if (clear_on_read)
      {
        delete ifxmit
        delete ifrecv
        delete ifmerged
      } 
    }
    I leave it up to anyone interested to write the appropriate userparameter scripts needed. :-) If anyone wants to also contribute a template as well that would be awesome.

    *NOTE* You, the end user of this script are responsible for any risk for use in production environments.
    Last edited by nelsonab; 18-10-2011, 03:11.
    RHCE, author of zbxapi
    Ansible, the missing piece (Zabconf 2017): https://www.youtube.com/watch?v=R5T9NidjjDE
    Zabbix and SNMP on Linux (Zabconf 2015): https://www.youtube.com/watch?v=98PEHpLFVHM

  • qix
    Senior Member
    Zabbix Certified SpecialistZabbix Certified Professional
    • Oct 2006
    • 423

    #2
    Nelsonab, this is just awesome!
    We can now start graphing network usage on a per application level.

    I need to try this out
    With kind regards,

    Raymond

    Comment

    • nelsonab
      Senior Member
      Zabbix Certified SpecialistZabbix Certified Professional
      • Sep 2006
      • 1233

      #3
      I've also added disk accounting.

      Code:
      # cat /proc/systemtap/zabbix/disk 
      COMMAND         DEV       READS  WRITES READ_KB WRITE_KB
      lxterminal      dm-0          8     460       0       7
      mysqld          dm-0          4     373      64    8454
      zabbix_server   dm-0          0     167       0      13
      packagekitd     dm-0         90      10      58       4
      sh              dm-0         88       0      87       0
      vmtoolsd        dm-0         80       0      12       0
      bash            dm-0         40       0       5       0
      cat             dm-0         19       0       9       0
      crond           dm-0         18       0      30       0
      dbus-daemon-lau dm-0         16       0      12       0
      firefox         dm-0          0      11       0       6
      ls              dm-0          9       0       4       0
      NetworkManager  dm-0          4       4       0       0
      zabbix_agentd   dm-0          0       5       0       0
      dbus-daemon     dm-0          4       0       0       0
      The latest version (zabbix.stp)
      Code:
      #SystemTap script for Zabbix
      #*NOTE* You, the end user of this script, are responsible for any risks #associated with using this script.
      #Compile string:
      # stap -m zabbix zabbix.stp
      #Information can be found in /proc/systemtap/zabbix/*
      #Current process statistics are found in network
      #clear_on_read sets weather or not the statistics are reset when read
      # the default is 0.
      #The network file is limited to 4096 bytes in size
      
      global ifxmit, ifrecv
      global io_read, io_write
      global merged
      global clear_on_read
      
      probe begin 
      {
        clear_on_read=0
      }
      
      function get_exec()
      {
        if (pid()==0)
          exec="LinuxKernel"
        else
          exec=execname()
        return exec
      }
      
      probe netdev.transmit
      {
        ifxmit[dev_name, get_exec()] <<< length
      }
      
      probe netdev.receive
      {
        ifrecv[dev_name, get_exec()] <<< length
      }
      
      probe vfs.read.return
      {
        if (($return>0) && (devname!="N/A"))
          io_read[devname,get_exec()] <<< $return
      }
      
      probe vfs.write.return
      {
        if (($return>0) && (devname!="N/A"))
          io_write[devname,get_exec()] <<< $return
      }
      
      probe procfs("clear_on_read").read
      {
        $value=sprintf("%u",clear_on_read)
      }
      
      probe procfs("clear_on_read").write
      {
        clear_on_read=strtol($value,2)
      }
      
      probe procfs("network").umask(0444).read.maxsize(4096)
      {
        delete merged
        $value=sprintf("%-15s %-7s %7s %7s %7s %7s\n",
               "COMMAND", "DEV", "XMIT_PK", "RECV_PK",
               "XMIT_KB", "RECV_KB")
      
        foreach ([dev, exec] in ifrecv) {
            merged[dev, exec] += @count(ifrecv[dev,exec]);
        }
        foreach ([dev, exec] in ifxmit) {
            merged[dev, exec] += @count(ifxmit[dev,exec]);
        }
        foreach ([dev, exec] in merged-) {
          n_xmit = @count(ifxmit[dev, exec])
          n_recv = @count(ifrecv[dev, exec])
          $value.=sprintf("%-15s %-7s %7d %7d %7d %7d\n",
                 exec, dev, n_xmit, n_recv,
                 n_xmit ? @sum(ifxmit[dev, exec])/1024 : 0,
                 n_recv ? @sum(ifrecv[dev, exec])/1024 : 0)
        }
      
        if (clear_on_read)
        {
          delete ifxmit
          delete ifrecv
        } 
        delete merged
      }
      
      probe procfs("disk").umask(0444).read.maxsize(4096)
      {
        delete merged
        $value=sprintf("%-15s %-7s %7s %7s %7s %7s\n",
               "COMMAND", "DEV", "READS", "WRITES",
               "READ_KB", "WRITE_KB")
      
        foreach ([dev, exec] in io_read) {
                merged[dev, exec] += @count(io_read[dev,exec]);
        }
        foreach ([dev, exec] in io_write) {
                merged[dev, exec] += @count(io_write[dev,exec]);
        }
        foreach ([dev, exec] in merged-) {
          writes = @count(io_write[dev, exec])
          reads = @count(io_read[dev, exec])
          $value.=sprintf("%-15s %-7s %7d %7d %7d %7d\n",
                 exec, dev, reads, writes,
                 reads ? @sum(io_read[dev, exec])/1024 : 0,
                 writes ? @sum(io_write[dev, exec])/1024 : 0)
        }
      
        if (clear_on_read)
        {
          delete ifxmit
          delete ifrecv
        } 
        delete merged
      }

      If you have the kernel module pre-compiled you can run it using
      Code:
      # staprun /path/to/zabbix.ko
      RHCE, author of zbxapi
      Ansible, the missing piece (Zabconf 2017): https://www.youtube.com/watch?v=R5T9NidjjDE
      Zabbix and SNMP on Linux (Zabconf 2015): https://www.youtube.com/watch?v=98PEHpLFVHM

      Comment

      • eterjack
        Member
        Zabbix Certified Specialist
        • Mar 2012
        • 60

        #4
        problem in systemtap

        stap -m zabbix zabbix.stp

        semantic error: missing x86_64 kernel/module debuginfo under '/lib/modules/3.2.0-33-generic/build' while resolving probe point kernel.function("vfs_write").return
        semantic error: no match while resolving probe point vfs.write.return

        Comment

        Working...