Using RRDTOOL to Graph IPMI data

From TDiWiki

IPMI data is readily available using a utility such as ipmitool. Unfortunately, the data returned from managed devices over IPMI can vary quite a bit in both content and format. Additionally, periodic calls to ipmitool are required to produce data that can be graphed over time. This article describes an example method to periodically collect data using ipmitool and then use RRDtool to aggregate and graph the data.

Summary

Create a ConsoleWorks Pseudo Console (requires a Pseudo Console license) to 1) run the polling script, 2) serve as a location for the polling script to log any interesting information such as collected data and/or errors, and 3) provide a Console name and user-URL links to the generated graphs.

Contents 1 Setting up the Pseudo Console files 2 Setting up ConsoleWorks 3 Verify script operation 4 Setting up graph URLs 5 Examples

[edit] Setting up the Pseudo Console files

• Create the /opt/ConsoleWorks/{invo}/pseudo_scripts directory, if needed

# cd /opt/ConsoleWorks/testinvo
# mkdir pseudo_scripts

• Create the Pseudo Console script link in the /opt/ConsoleWorks/{invo}/pseudo_consoles directory

# cd /opt/ConsoleWorks/testinvo/pseudo_consoles
# echo /opt/ConsoleWorks/testinvo/pseudo_scripts/ipmiinfo.sh %c %u %p >ipmiinfo
# chmod 644 ipmiinfo

• Create the polling script

# cd ../pseudo_scripts
# cat >ipmiinfo.sh
#!/bin/bash

WKD=/opt/ConsoleWorks/testinvo
PAS="$3"
USR="$2"  ##PUT SHELL script here to PARSE this 'username' field if you cannot assume CONSOLENAME == HOSTNAME
CON="$1"

IPMILAN=lanplus

#Attempt to use ipmi v2.0 FIRST
ipmitool -U "$USR" -P "$PAS" -H "$CON" -I $IPMILAN chassis status 2>/dev/null
if [ $? != 0 ]
then
  IPMILAN=lan
  ipmitool -U "$USR" -P "$PAS" -H "$CON" -I $IPMILAN chassis status 2>/dev/null
fi

while [ true ]
do
echo "Start collection"
  ipmitool -U "$USR" -P "$PAS" -H "$CON" -I $IPMILAN sensor | perl -e '

my @colors = (
  "F00000", "C00000", "A00000", "800000",
  "00F000", "00C000", "00A000", "008000",
  "0000F0", "0000C0", "0000A0", "000080" );
my $wkdir = shift @ARGV;
my $cons = shift @ARGV;
my $a,$ix=0;
my @files = ();
printf("Data collection starts for IPMI temp info, console $cons\n");
while ($a=<>) {
  chomp($a);
  next unless ($a=~/(.*temp.*ok|.*rpm.*ok)/i); #Pick up both TEMP and FAN info
  $ix++;
  $a =~ s/ //g;
  $a =~ s/\///g;
  my @flds = split("\\|",$a);
  my $dataname = $cons."-".$ix."-".$flds[0];  #e.g. T5120-6-B3CH1D0TEMP
  push(@files,$dataname);
  printf("Collected data: %s = %s\n",$dataname,$flds[1]);
  system(sprintf("%s/info_graph_update.sh %s %s %u",$wkdir,$dataname,$flds[1],time()));
}


my $dix0;
my $dix1;
my $dix2;
my $r;

$dix0=0;
$dix1=0;
$dix2=0;
my @plottemp = (
  "rrdtool graph $wkdir/html/$cons"."_temp.png --start end-1h --width 1200 --height 800 \\",
  "  --vertical-label Temperature \\",
  "  --title Memory\\ Temperature\\ Graph\\ -\\ Past\\ Hour \\",
  "  --no-gridfit --slope-mode \\",
  (map { $r=sprintf(
  "  DEF:temp%02d=$wkdir/graph-%s.rrd:Temperature:AVERAGE \\", $dix0, $_);
      $dix0++; $r;
    } grep /temp/i,@files ),
  (map { $r=sprintf(
  "  VDEF:t%02dmax=temp%02d,MAXIMUM VDEF:t%02davg=temp%02d,AVERAGE VDEF:t%02dmin=temp%02d,MINIMUM \\", $dix1,$dix1,$dix1,$dix1,$dix1,$dix1);
      $dix1++; $r;
    } grep /temp/i,@files ),
  (map { $r=sprintf(
  "  LINE2:temp%02d#%02s:\"Sensor %s\" \\", $dix2, $colors[$dix2] , $_);
      $dix2++; $r;
    } grep /temp/i,@files ),
  " >/dev/null 2>/dev/null");

$dix0=0;
$dix1=0;
$dix2=0;
my @plotrpm = (
  "rrdtool graph $wkdir/html/$cons"."_rpm.png --start end-1h --width 1200 --height 800 \\",
  "  --vertical-label Fan\\ RPM \\",
  "  --title Fan\\ RPM\\ Graph\\ -\\ Past\\ Hour \\",
  "  --no-gridfit --slope-mode \\",
  (map { $r=sprintf(
  "  DEF:temp%02d=$wkdir/graph-%s.rrd:Temperature:AVERAGE \\", $dix0, $_);
      $dix0++; $r;
    } grep /rpm/i,@files ),
  (map { $r=sprintf(
  "  VDEF:t%02dmax=temp%02d,MAXIMUM VDEF:t%02davg=temp%02d,AVERAGE VDEF:t%02dmin=temp%02d,MINIMUM \\", $dix1,$dix1,$dix1,$dix1,$dix1,$dix1);
      $dix1++; $r;
    } grep /rpm/i,@files ),
  (map { $r=sprintf(
  "  LINE2:temp%02d#%02s:\"Sensor %s\" \\", $dix2, $colors[$dix2] , $_);
      $dix2++; $r;
    } grep /rpm/i,@files ),
  " >/dev/null 2>/dev/null");

system(join("\n",@plottemp).$/);
system(join("\n",@plotrpm).$/);

' "$WKD" "$1"
  sleep 20
done
^D
# chmod a+x ipmiinfo.sh

• Create the graph update script

# cd ..
# cat >info_graph_update.sh
#!/bin/bash
#
# $1 = datafile basename
# $2 = current floating point value
# $3 = collected ctime
#

#echo Update $1 data point $2
WKD=/opt/ConsoleWorks/Crotalus
NOW=$3 #(perl -e 'print time();')

if [ ! -f $WKD/graph-$1.rrd ] ; then
  rrdtool create $WKD/graph-$1.rrd --step 20 DS:Temperature:GAUGE:900:U:U RRA:AVERAGE:0.75:1:256 RRA:AVERAGE:0.75:3:256 RRA:AVERAGE:0.75:10:256 RRA:AVERAGE:0.75:24:256 RRA:AVERAGE:0.75:48:256
fi

rrdtool update $WKD/graph-$1.rrd $NOW:$2
^D
# chmod a+x info_graph_update.sh

[edit] Setting up ConsoleWorks

• Configure a Pseudo Console to use the new ipmiinfo script
  • Add the Console with a NAME that equals the HOST you will poll via IPMI
  • Select connector type: PSEUDO
  • Supply the username that will be used to authenticate over IPMI
  • Supply the password that will be used to authenticate over IPMI

[edit] Verify script operation

• Examine the console output for error messages and make corrections if needed
• Examine /opt/ConsoleWorks/{invo} for files of the form 'graph-{consolename}-{something}.rrd'
• Examine /opt/ConsoleWorks/{invo}/html/ for graph files of the form '{consolename}*.png'

[edit] Setting up graph URLs

• On the Console's Edit page, enter User URLs for the graphs found in the previous step
  • e.g., http://consoleworksserver.mycompany.com:5176/{consolename}_temp.png
  • e.g., http://consoleworksserver.mycompany.com:5176/{consolename}_rpm.png

[edit] Examples

• FAN RPM Graph

• Temperature Graph