RRDTool is a free, open source and powerful tool that is awesome for time-series data like sensor graphing. It is both a database and a graph creation tool. Graphs can be simple, or very complex. Whether you know it or not, you have probably seen many graphs that were created using RRDTool as it has been a mainstay graphing tool for more than a decade. Check out the links:

I highly recommend reviewing the online RRDTutorial to gain an understanding of how it works and the concepts involved.

In this article I am going to show you how to pull down your sensor data from a Dia application from the iDigi Device Cloud, parse the XML, update the RRDTool database and create a simple graph. I will be doing this in Linux, so you will need to be at least competent in Linux, with basic knowledge of how to setup a Linux server with a web server, basic shell scripting and basic HTML creation. You do NOT have to be a programmer (other than basic shell scripting skills).

I am going to keep this simple and not make it a full tutorial on RRDTool, the iDigi Device Cloud or Linux. There is plenty of documentation, including some great tutorials on the RRDTool website, as well as a wealth of examples on the web that can be found using your favorite search engine. Hopefully this will give you a jumpstart with basic graphing and you can go from there.

To start, here are the tools we are going to use:

• A Linux server — I use a Rackspace cloud server based on Ubuntu server
• A web server — I use Apache, but feel free to use your preferred web server; the web server needs are basic
• Cron to run the update script on a regular basis
• Wget — I use this as a simple “web services client.” Basically we will use it to pull the XML data
• XMLstarlet — A command line utility to inspect and parse data from an XML file
• Basic GNU Linux tools like “date” and “sed”
• RRDTool — This is both a database and graph creation tool

Pulling the Data
Step one is to pull the data down from the iDigi Device Cloud. I am going to use wget to do this, as RESTful web services are really just URLs and wget is good at this. There are other tools like “curl” that will do the job as well.

Here is a simple wget command line (all on one line, remove the ‘\’):

/usr/bin/wget --user=youruserid --password=yourpassword \
https://developer.idigi.com/ws/DiaChannelDataFull

This will return an XML file “./DiaChannelDataFull”. This XML file has all the latest sensor readings for all the devices associated with your iDigi developer account.

Parsing a Sensor Measurement from the XML File

Here we use a tool called XMLstarlet that should be available in the package repository for most Linux distributions. XMLstarlet is a simple command line tool to parse an XML file.

Here is a simple xmlstarlet command line to pull the temperature from sensor0.temperature (all on one line, remove the ‘\’s):

xmlstarlet sel -t -m \
"/result/DiaChannelDataFull/id[devConnectwareId='00000000-00000000-00409DFF-FF999999' \
and ddInstanceName='sensor0' and dcChannelName='temperature']" \
-v ../dcdFloatValue DiaChannelDataFull

You will need to correct the ConnectwareID to match your device. You can get this by inspecting the XML file with less or cat. This line should return the current temperature from sensor0.temperature (adjust as necessary for your situation).

These two lines can go in your bash script to pull out the temperature ($sensorvalue) and timestamp ($iso8601time) and assign them to variables for use later in the script:

sensorvalue=`xmlstarlet sel -t -m \
"/result/DiaChannelDataFull/id[devConnectwareId='00000000-00000000-00409DFF-FF999999' \
and ddInstanceName='sensor0' and dcChannelName='temperature']" \
-v ../ dcdFloatValue DiaChannelDataFull`

iso8601time=`xmlstarlet sel -t -m
"/result/DiaChannelDataFull/id[devConnectwareId='00000000-00000000-00409DFF-FF999999' \
and ddInstanceName='sensor0' and dcChannelName='temperature']" \
-v ../ dcdUpdateTime DiaChannelDataFull`

Converting Timestamp for Use in RRDTool
iDigi returns timestamps in ISO 8601 format. For example: 2011-11-27T01:09:56Z. RRDTool wants timestamps in epoch time, which is the number of seconds since Midnight 1/1/1990 GMT. There was no easy way I could find to convert ISO time to epoch time. I came up with the slightly convoluted method of using sed to convert to a format the “date” command can use, then used date to convert to epoch time.

# Step 1 is to convert the ISO timestamp to something the Linux date command can read.
# The result is changing "2011-11-27T01:09:56Z" to "2011-11-27 01:09:56 +0000"
datecompatibletime=`echo $iso8601time | sed s/T/' '/ | sed s/Z/' +0000'/`

# The next line uses the date command to convert to epoch time.
#The result is changing "2011-11-27 01:09:56 +0000" to "1322356196"
epochtime=`date +%s -d "$datecompatibletime"`

The result is that variable $epochtime contains the timestamp in Unix epoch time.

Creating a Database Using RRDTool
Now we have our temperature ($sensorvalue) and a timestamp compatible with RRDTool ($epochtime) so we are almost ready to update the rrd database with those values.

First, we need to create the database. This is a one-time step. You will likely want to refer to the tutorials and documentation on the RRDTool website for all the options here, but for now I am going to create a database that keeps 5 minute values for 1 day, 15 minute averages for 1 week and 1 hour averages for 1 year.

Here is a sample command line to create the database. Remember, this is a one-time operation, so it should not be part of your cron script!

rrdtool create sensor0.temperature.rrd --start "now - 8h" --step=300 \
DS:tempC:GAUGE:1000:U:U \
RRA:AVERAGE:0.5:1:288 \
RRA:AVERAGE:0.5:3:672 \
RRA:AVERAGE:0.5:12:8760

This will create a file called sensor0.temperature.rrd. This is the RRD database. Note that the initial database is filled with null values, so it will never grow in size. Do not worry about it filling up your filesystem over time!

Updating the RRDTool Database with Sensor Values
Now that we have our database created, we are ready to update it. This line is very simple. It is simply updating the database we created with the latest values, using the variables we created earlier in our script:

rrdtool update sensor0.temperature.rrd $epochtime:$sensorvalue

Creating a Simple Graph
Now it’s time for the fun part, creating the graph. Again, we are going to keep it simple. Refer to the RRDTool doc and samples on the web for more complex graphing help.

This command line will create a graph in /var/www/ (adjust as necessary for your webserver’s document path) with the temperature over the last 2 days.

/usr/bin/rrdtool graph /var/www/graphs/temp_last2days.png \
-t "Temperature - Last 2 Days" \
-v "Degress Celsius"  -F --start -2d  --end now \
DEF:temp=sensor0.temperature.rrd:tempC:AVERAGE \
LINE2:temp#FF0000:"Temperature"

You can change the duration of the graph by changing “-start 2d” to something like “-start 12d” for 12 days.

You will likely want to wrap this up in HTML and place your graphs in a nice web page — I will leave that up to you. Here is some simple HTML to show the graph:

<p>Here is my cool new graph:</p>
<img src="/graphs/temp_last2days.png">
<p>I hope you like it!</p>

Creating Your Script to Run in Cron
I have given you a bunch of code snippets, now it is time to put it all together in a bash script. I put together this sample script that you can add to cron and run on a regular interval. You will likely need to modify paths for your system and change the sensor parameters to pull the data points you want.

#!/bin/bash

# Change these values to match your system and modify graph properties

deviceid="00000000-00000000-00409DFF-FF999999"
diainstance="sensor0"
diachannel="temperature"
idigicloudurl="https://developer.idigi.com"
idigiuserid="idigiusername"
idigipw="idigipassword"
graphfile="/var/www/graph/temp_last2days.png"
graphtitle="Temperature - Last 2 Days"
graphtype="LINE2"
graphlinecolor="#FF0000"
graphbgcolor="#FFFFFF"
graphfontcolor="#000000"
graphvaxis="Degrees Celsius"
graphunits="C"
graphsensorlabel="Temperature"
graphstart="-2d"
# Note:  rraname must be the same as the rra name you used when creating the RRD database!
rraname="tempC"
rrddatabase="/home/rrdtool/graph_temp/sensor0.temperature.rrd"
workdir="/home/rrdtool/graph_temp/"

# You should not need to adjust much below this line.
# Paths to executables might need to be adjusted to match your system.

cd $workdir

rm ./DiaChannelDataFull

/usr/bin/wget --user=$idigiuserid --password=$idigipw \
$idigicloudurl/ws/DiaChannelDataFull

# Parse the XML, grabbing the sensor value and the timestamp
sensorvalue=`xmlstarlet sel -t -m \
"/result/DiaChannelDataFull/id[devConnectwareId='$deviceid' \
and ddInstanceName='$diainstance' and dcChannelName='$diachannel']" \
-v ../dcdFloatValue DiaChannelDataFull`

iso8601time=`xmlstarlet sel -t -m \
"/result/DiaChannelDataFull/id[devConnectwareId='$deviceid' \
and ddInstanceName='$diainstance' and dcChannelName='$diachannel']" \
-v ../dcdUpdateTime DiaChannelDataFull`

# We will convert the timestamp from ISO format to Unix Epoch format in 2 steps.
# Step 1 is to convert the ISO timestamp to something the Linux date command can read.
# The result is changing "2011-11-27T01:09:56Z" to "2011-11-27 01:09:56 +0000"
datecompatibletime=`echo $iso8601time | sed s/T/' '/ | sed s/Z/' +0000'/`

# The next step uses the date command to convert to epoch time.
# The result is changing "2011-11-27 01:09:56 +0000" to "1322356196"
epochtime=`date +%s -d "$datecompatibletime"`

# Uncomment the following 2 lines to output some debug info to the console
# echo "Sensor value: $sensorvalue ; ISO8601 time: $iso8601time"
# echo "date compatible time: $datecompatibletime ; epochtime: $epochtime"

# Post the sensor value to the RRDTool database
/usr/bin/rrdtool update $rrddatabase $epochtime:$sensorvalue

# Set a variable with server time info for use in graph below
timedatestring=`date +%F\ %H\\\:%M\\\:%S\ %Z`

# Create the graph
/usr/bin/rrdtool graph $graphfile \
-c BACK$graphbgcolor -c FONT$graphfontcolor \
-t "$graphtitle" \
-v "$graphvaxis"  -F --start $graphstart  --end now \
COMMENT:" \l" \
COMMENT:"-------------------------------------------\l" \
COMMENT:"SENSOR         MIN    MAX    AVG    LAST   \l" \
COMMENT:"-------------------------------------------\l" \
DEF:value1=$rrddatabase:$rraname:AVERAGE \
$graphtype:value1$graphlinecolor:"$graphsensorlabel" \
VDEF:value1max=value1,MAXIMUM \
VDEF:value1min=value1,MINIMUM \
VDEF:value1avg=value1,AVERAGE \
VDEF:value1las=value1,LAST \
GPRINT:value1min:"%2.0lf $graphunits " \
GPRINT:value1max:"%2.0lf $graphunits " \
GPRINT:value1avg:"%2.0lf $graphunits " \
GPRINT:value1las:"%2.0lf $graphunits \l" \
COMMENT:"Generated on\: $timedatestring\r"

Sample Graphs
The script above will create a graph something like this:

With a few quick changes, you can change the colors and graph type. Change these configuration variables:

graphtype="AREA"
graphlinecolor="#D2691E"
graphbgcolor="#000000"
graphfontcolor="#7FFF00"

and you will get a graph like this:

RRDTool has many graph options, I have just demonstrated a few simple ones. Read the RRDTool doc and examples for more ideas.

For a listing of color codes, use an Internet search engine and search for “html color chart”.

Summary
I hope you found this interesting and give it a try. RRDTool is an awesome tool for use with iDigi sensor data. Once you get familiar with RRDTool you can do some really cool things. Again, I highly recommend the RRDTool tutorial, then digging into the various graph options to make your graphs more functional and cool looking.

Have fun!

Meet Us in Nice
— WaveForum is Coming to France —

Digi is bringing the WaveForum Developers’ Conference to the heart of the Riviera. Join us in Nice, France for two days of expert education focused on the latest advances in wireless M2M device networking. Digi will offer many technical sessions covering a range of topics, including:

• Setting up a mesh network
• Protocol selection
• Web/Smartphone applications
• Using wireless WAN networks
• AMI/AMR wireless solutions
• iDigi® Device Cloud™ for M2M networks

Dates: February 14-15, 2012

Location: Radisson Blu Hotel, Nice, France
Cost:199.00

On the Blog
— Projects, News & Notes —

XIG Ego Ticker — Matt Richardson, host of Make: Live, a Masters student at ITP and a notable technophile, is using Digi’s XBee® Internet Gateway to track and display his online scores (Google results, YouTube followers, etc.) on a big LED display he calls the Ego Ticker. More >

CheerLights Project — This social networking project allows people from across the globe to synchronize their holiday lights. Find out how to make a CheerLights controller with Arduino, XBee® and Digi ConnectPort® X2. Learn more about the project by visiting the iDigi blog, the CheerLights site or Twitter.

International CES — There were a lot of innovative products and exciting technologies at this year’s International Consumer Electronics Show. Here’s a few of our favorites:

• The Marvell chip and Smart Appliances Platform is making household items network-aware.

• WiMM Lab’s new class of Android-based smart watches

• The Escort Live speed-trap alert system won a 2012 Innovations Design and Engineering Award at CES.

• BiKN is an iPhone geo-location app system that finds tags you attach to the stuff you care about (keys, wallet, etc.).

Share Your Projects & Ideas — There are lots of great applications that use the iDigi Device Cloud and we want to hear about yours. Contact the team at thunderheads@digi.com to share us your ideas and stories.