Ad Widget

Collapse

Is there a way to collect any IPMI sensor data?

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • blind_oracle
    Junior Member
    • Feb 2009
    • 23

    #1

    Is there a way to collect any IPMI sensor data?

    There were a number of such topics, in Russian particulary, but so far no good.

    IPMI in zabbix now collects only a small fraction of sensors, such as temperatures and, sometimes, FAN speeds, voltage.

    Other readings, for example from iLO: Power Meter, Power Suppy status etc cannot be read into zabbix directly.

    But ipmitool shows them correcty:
    Code:
    # ipmitool -I lanplus -H 10.1.1.1 -U zabbix -P xxxxxx -L user sdr
    UID Light        | 0 unspecified     | ok
    Int. Health LED  | 0 unspecified     | ok
    Ext. Health LED  | 0 unspecified     | ok
    Power Supply 1   | 0 unspecified     | nc
    Power Supply 2   | 0 unspecified     | nc
    Power Supplies   | 0 unspecified     | nc
    VRM 1            | 0 unspecified     | cr
    VRM 2            | 0 unspecified     | cr
    Fan 1            | 45.08 unspecifi | nc
    Fan 2            | 45.08 unspecifi | nc
    Fan 3            | 41.94 unspecifi | nc
    Fan 4            | 36.06 unspecifi | nc
    Fan 5            | 36.06 unspecifi | nc
    Fan 6            | 36.06 unspecifi | nc
    Fans             | 0 unspecified     | nc
    Temp 1           | 40 degrees C      | ok
    Temp 2           | 18 degrees C      | ok
    Temp 3           | 45 degrees C      | ok
    Temp 4           | 45 degrees C      | ok
    Temp 5           | 52 degrees C      | ok
    Temp 6           | 45 degrees C      | ok
    Temp 7           | 45 degrees C      | ok
    Power Meter      | 358 Watts         | cr
    What can we do about it? Other than using external check with `ipmitool`...
  • richlv
    Senior Member
    Zabbix Certified Trainer
    Zabbix Certified SpecialistZabbix Certified Professional
    • Oct 2005
    • 3112

    #2
    first guess - https://support.zabbix.com/browse/ZBXNEXT-300
    Zabbix 3.0 Network Monitoring book

    Comment

    • blind_oracle
      Junior Member
      • Feb 2009
      • 23

      #3
      So, these patches are not yet accepted? Is there any planned support for discrete sensors in upcoming releases, such as 1.9/2.0?

      Comment

      • richlv
        Senior Member
        Zabbix Certified Trainer
        Zabbix Certified SpecialistZabbix Certified Professional
        • Oct 2005
        • 3112

        #4
        as far as i know, there are no short term plans regarding discrete ipmi sensor support
        Zabbix 3.0 Network Monitoring book

        Comment

        • blind_oracle
          Junior Member
          • Feb 2009
          • 23

          #5
          Okay, then i'll have to stick parsing ipmitool output, thanks

          Comment

          • CeeEss
            Senior Member
            Zabbix Certified Specialist
            • Nov 2007
            • 103

            #6
            Originally posted by richlv
            as far as i know, there are no short term plans regarding discrete ipmi sensor support
            I'm confused: what then is the intent of Items with type IPMI Agent? Tried to set a sensor "System Temp" for a Supermicro host/device which works fine when polled using ipmitool but never got any data back and no errors in serverlog. impitool returms a lot information more than simply the value fo the sensor, of course, but how does one use Item type IPMI Agent?

            Comment

            • blind_oracle
              Junior Member
              • Feb 2009
              • 23

              #7
              Yes, IPMI is everywhere, and Zabbix somehow is not fast at supporting it fully.

              Below is the script that i use to obtain the IPMI values on my Zabbix server.
              It uses host's macro {$IPMI_HOSTNAME}, ipmitool to obtain numeric values, and ipmi-sensors (from freeipmi) to get discrete values. I've failed to use only one tool for both, as one returns good values for discrete values, and other - for numeric.
              The script caches values for $expires seconds in a file before retrieving them again.
              Use in zabbix 'External Check' item, such as: ipmi_sensors.pl['Fan 1' numeric {$IPMI_HOSTNAME}]
              And don't forget to put this script into directory defined by ExternalScripts zabbix config value.
              Code:
              #!/usr/bin/perl -w
              
              use strict;
              use warnings;
              use Fcntl ':flock';
              
              my $sensor = $ARGV[1];
              my $type = $ARGV[2];
              my $server = $ARGV[3];
              $type = 'numeric' if not defined $type;
              exit(1) if not defined $server or not defined $sensor;
              
              my $expires = 60;
              
              my $user = 'zabbix';
              my $pass = 'xxxxxx';
              
              my $ipmi_cmd = '';
              my $cache_file = '/var/tmp/ipmi_sensors_'.$server;
              if($type eq 'discrete') {
                  $ipmi_cmd = '/opt/freeipmi/sbin/ipmi-sensors -D LAN2_0 -h '.$server.' -u '.$user.' -p '.$pass.' -l USER --no-header-output --comma-separated-output 2>/dev/null';
                  $cache_file .= '_discrete';
              } elsif($type eq 'numeric') {
                  $ipmi_cmd = '/usr/bin/ipmitool -I lanplus -H '.$server.' -U '.$user.' -P '.$pass.' -L user -A MD5 sdr -c 2>/dev/null';
                  $cache_file .= '_numeric';
              }
              
              my @rows = ();
              if(not -e $cache_file) {
                  open(CACHE, '>>', $cache_file);
                  if(flock(CACHE, LOCK_EX | LOCK_NB)) {
                      my $results = results();
                      if(defined $results) {
                          truncate(CACHE, 0);
                          print CACHE $results;
                          close(CACHE);
                      } else {
                          close(CACHE);
                          unlink($cache_file);
                          exit(1);
                      }
                  }
              } else {
                  open(CACHE, '>>', $cache_file);
                  if(flock(CACHE, LOCK_EX | LOCK_NB)) {
                      my @stat = stat($cache_file);
                      my $delta = time() - $stat[9];
                      if($delta > $expires or $delta < 0) {
                          my $results = results();
                          if(defined $results) {
                              truncate(CACHE, 0);
                              print CACHE $results;
                              close(CACHE);
                          } else {
                              close(CACHE);
                              unlink($cache_file);
                              exit(1);
                          }
                      }
                  }
              }
              
              open(CACHE, '<' . $cache_file);
              flock(CACHE, LOCK_EX);
              @rows = <CACHE>;
              close(CACHE);
              
              foreach my $row (@rows) {
                  my @cols = split(',', $row);
              
                  if($type eq 'discrete') {
                      if($cols[1] eq $sensor) {
                          $cols[5] = substr($cols[5], 1);
                          chop($cols[5]);
                          chop($cols[5]);
                          print $cols[5] . "\n";
                      }
                  } elsif($type eq 'numeric') {
                      if($cols[0] eq $sensor) {
                          if($cols[1] eq '') {
                              print "0\n";
                          } else {
                              print $cols[1] . "\n";
                          }
                      }
                  }
              }
              
              sub results {
                  my $results = `$ipmi_cmd`;
                  if($? == 0) {
                      return $results;
                  } else {
                      return undef;
                  }
              }

              Comment

              • CeeEss
                Senior Member
                Zabbix Certified Specialist
                • Nov 2007
                • 103

                #8
                Brilliant - thank you, Oracle, i'll give it a try. I've read the section on IPMI monitoring in the (very good) Zabbix book, checked my configuration using ipmitool but still cannot collect data. I get network errors, followed by "Disabling IPMI host [hostIP]" in the serverlog, so it would seem that Zabbix might be running IPMI checks on the wrong host address? Q: If i have both a host IP/DNS _and_ an IPMI address configured for a host, does Zabbix adapt to use the IPMI address for IPMI checks? Is Host IPMI configuration exclusive of the standard host config ie: you can use one or the other but not both?

                thanks

                Comment

                • richlv
                  Senior Member
                  Zabbix Certified Trainer
                  Zabbix Certified SpecialistZabbix Certified Professional
                  • Oct 2005
                  • 3112

                  #9
                  Originally posted by CeeEss
                  Q: If i have both a host IP/DNS _and_ an IPMI address configured for a host, does Zabbix adapt to use the IPMI address for IPMI checks?
                  it should use ipmi address for items of ipmi type only.

                  things to check :

                  a) is ipmi support compiled in ?
                  b) against which version of openipmi is zabbix compiled ? 2.0.2 is known to work, 2.0.7 & 2.0.13 are known not to work, 2.0.14 & 2.0.16 are known to work again...
                  c) how many ipmi pollers are started (config param in server config file) ?
                  Zabbix 3.0 Network Monitoring book

                  Comment

                  • blind_oracle
                    Junior Member
                    • Feb 2009
                    • 23

                    #10
                    I'm also getting network errors with Zabbix's IPMI agent, the host gets disabled frequently, and then enabled again. The values however are being collected, but not at intervals, defined in Item due to these network errors.

                    And of course only numeric values are collected (fan speed, temperature), discreet, as mentioned up in the topic, are not supported by Zabbix at all.

                    So i'll be converting all my native IPMI Zabbix templates to external check by this script, as it works without errors so far.

                    Comment

                    • CeeEss
                      Senior Member
                      Zabbix Certified Specialist
                      • Nov 2007
                      • 103

                      #11
                      Originally posted by richlv
                      it should use ipmi address for items of ipmi type only.

                      things to check :

                      a) is ipmi support compiled in ?
                      b) against which version of openipmi is zabbix compiled ? 2.0.2 is known to work, 2.0.7 & 2.0.13 are known not to work, 2.0.14 & 2.0.16 are known to work again...
                      c) how many ipmi pollers are started (config param in server config file) ?
                      a) yes:. From config.log:

                      $ ./configure --enable-server --enable-agent --with-mysql --with-net-snmp --with-openipmi

                      b) OpenIPMI-2.0.16-12.el6.x86_64
                      c) 1 (i'm only testing one sensor on a single server as proof of concept)

                      thanks!

                      Comment

                      • CeeEss
                        Senior Member
                        Zabbix Certified Specialist
                        • Nov 2007
                        • 103

                        #12
                        and ... is {$IPMI_HOSTNAME} a valid macro given it is defined as an IP address? No disrespect to blind_oracle

                        Comment

                        • richlv
                          Senior Member
                          Zabbix Certified Trainer
                          Zabbix Certified SpecialistZabbix Certified Professional
                          • Oct 2005
                          • 3112

                          #13
                          Originally posted by CeeEss
                          a) yes:. From config.log:

                          $ ./configure --enable-server --enable-agent --with-mysql --with-net-snmp --with-openipmi
                          just to be completely sure, when zabbix server starts up, does it print "YES" for ipmi in the logfile ?
                          Zabbix 3.0 Network Monitoring book

                          Comment

                          • blind_oracle
                            Junior Member
                            • Feb 2009
                            • 23

                            #14
                            Here's new version of script, unified to use "ipmi-sensors" only, without ipmitool.

                            Code:
                            #!/usr/bin/perl -w
                            
                            use strict;
                            use warnings;
                            use Fcntl ':flock';
                            
                            my $sensor = $ARGV[1];
                            my $type = $ARGV[2];
                            my $server = $ARGV[3];
                            $type = 'numeric' if not defined $type;
                            exit(1) if not defined $server or not defined $sensor;
                            
                            my $expires = 60;
                            
                            my $user = 'zabbix';
                            my $pass = 'xxxxxxxxx';
                            
                            my $ipmi_cmd = '';
                            my $cache_file = '/var/tmp/ipmi_sensors_'.$server;
                            $ipmi_cmd = '/opt/freeipmi/sbin/ipmi-sensors -D LAN2_0 -h '.$server.' -u '.$user.' -p '.$pass.' -l USER -W discretereading --no-header-output --comma-separated-output 2>/dev/null';
                            
                            my @rows = ();
                            if(not -e $cache_file) {
                                open(CACHE, '>>', $cache_file);
                                if(flock(CACHE, LOCK_EX | LOCK_NB)) {
                                    my $results = results();
                                    if(defined $results) {
                                        truncate(CACHE, 0);
                                        print CACHE $results;
                                        close(CACHE);
                                    } else {
                                        close(CACHE);
                                        unlink($cache_file);
                                        exit(1);
                                    }
                                }
                            } else {
                                open(CACHE, '>>', $cache_file);
                                if(flock(CACHE, LOCK_EX | LOCK_NB)) {
                                    my @stat = stat($cache_file);
                                    my $delta = time() - $stat[9];
                                    if($delta > $expires or $delta < 0) {
                                        my $results = results();
                                        if(defined $results) {
                                            truncate(CACHE, 0);
                                            print CACHE $results;
                                            close(CACHE);
                                        } else {
                                            close(CACHE);
                                            unlink($cache_file);
                                            exit(1);
                                        }
                                    }
                                }
                            }
                            
                            open(CACHE, '<' . $cache_file);
                            flock(CACHE, LOCK_EX);
                            @rows = <CACHE>;
                            close(CACHE);
                            
                            foreach my $row (@rows) {
                                my @cols = split(',', $row);
                            
                                if($cols[1] eq $sensor) {
                                    if($type eq 'discrete') {
                                        my $r = $cols[5];
                                        $r =~ s/\'//g;
                                        chop($r);
                                        print $r . "\n";
                                    } elsif($type eq 'numeric') {
                                        if($cols[3] eq '') {
                                            print "0\n";
                                        } else {
                                            print $cols[3] . "\n";
                                        }
                                    }
                                }
                            }
                            
                            sub results {
                                my $results = `$ipmi_cmd`;
                                if($? == 0) {
                                    return $results;
                                } else {
                                    return undef;
                                }
                            }
                            PS:
                            1. You don't need IPMI support in zabbix to use external script.
                            2. You need IPMI_HOSTNAME macro only when using external script.

                            Comment

                            • mikesjn
                              Junior Member
                              • Jan 2009
                              • 11

                              #15
                              script works, not seeing anything in Zabbix

                              Hi,
                              Zabbix 2.0.5 on SLES 11 SP2

                              If I run the script as the zabbix user I get a return ie:

                              ************************************************
                              zabbix@zabbixsvr:/usr/local/share/zabbix/externalscripts> ./ipmi_sensors.pl 'Fan1 RPM' numeric 192.168.128.175
                              2760.00
                              ************************************************

                              If I create an external check,
                              key: ipmi_sensors.pl['Fan1 RPM',numeric,192.168.128.175]
                              Type of Info: Numeric (unsigned)
                              Data type: Decimal
                              Units: rpm

                              I get this in the error log:

                              11494:20130409:112520.154 item [Dell server:ipmi_sensors.pl['Fan1 RPM',numeric,193.150.132.175]] became not supported: Received value [] is not suitable for value type [Numeric (unsigned)] and data type [Decimal]


                              Same for numeric float.

                              Host interface shows correct IPMI interface ( I can get a return using inbuilt IPMI - I want to start using discrete values though - these just don't work as using text it is getting no return at all)

                              Anyone got any ideas?

                              Comment

                              Working...