Trampoline Systems is hosting a visualisation workshop this coming Saturday, 24th May, organised by the Open Knowledge Foundation. Come along if you’re interested in open source visualisation technologies (Prefuse, Flare etc.). The goal is to have a very informal setting to talk about various aspects of visualising data. Find out more in the official announcement.

Hope to see you here!

No big changes this time, mainly compatibility fixes for JRuby 1.0. It is now also possible to build the project using Maven, for those too afraid to use rake. Documentation and code for springy are available here.

I’m also happy to announce that Craig Walls, the author of “Spring in Action”, is going to talk about Springy as part of his “Spring Cleaning: Tips for Managing XML Clutter” talk at this year’s No Fluff Just Stuff series of events as well as the Spring Experience 2007 in Florida.

We use last.fm a lot in the office - but one thing I always found annoying was that there’s no easy way to find out what’s currently playing (you need to go to the web page and hit refresh, very distracting) so I knocked up a little ruby script which uses growl to display the currently playing song.

#!/usr/bin/env ruby
require ‘rubygems’
require ‘ruby-growl’
require ‘hpricot’
require ‘cgi’
require ‘open-uri’

raise “#{$0} <user>” if ARGV.empty?
user = ARGV[0]
already_notified = []

while true do
now_listening = Hpricot(open(“http://www.last.fm/user/#{user}”))/“.nowListening td.subject”
unless now_listening.empty?
currently_played = now_listening.last.inner_text.strip
unless already_notified.include?(currently_played)
already_notified << currently_played
g = Growl.new(“localhost”, “ruby-growl”, [“ruby-growl Notification”])
g.notify(“ruby-growl Notification”, “#{user} is listening to”,
CGI::unescapeHTML(currently_played))
end
end
sleep 60
end

Make sure that you have hpricot and ruby-growl installed (gem install hpricot ruby-growl -y), then copy and paste the script (or download it) and start it with a last.fm username as parameter. You also need to have “Listen for incoming notifications” and “Allow remote registration” checked in growl’s preferences (System Preferences | Growl | Network).

While working on some tweaks to our website I decided that for the archive I needed to be able to select a list of categories used. Should be easy, right? Here’s a sample of the XML:

<machines-archive section="weblog" section-id="3">
<entry id="78" handle="installing-oracle-10g-on-64-bit-centos-5" linked-count="0">
<date year="2007" month="06" date="28" weekday="4">2007-06-28</date>
<time hour="12" minute="35">12:35</time>
<author />
<fields>
<title handle="installing-oracle-10g-on-64-bit-centos-5">installing Oracle 10g on 64 bit centos 5</title>
<categories>
<item handle="ruby">Ruby</item>
<item handle="java">Java</item>
<item handle="programming">Programming</item>
<item handle="code">Code</item>
</categories>
</fields>
<comments count="0" spam="0" />
</entry>
<entry id="75" handle="compulsory-wearing-of-facial-hair" linked-count="0">
<date year="2007" month="06" date="14" weekday="4">2007-06-14</date>
<time hour="14" minute="15">14:15</time>
<author />
<fields>
<title handle="compulsory-wearing-of-facial-hair">compulsory wearing of facial hair</title>
<categories>
<item handle="programming">Programming</item>
</categories>
</fields>
<comments count="0" spam="0" />
</entry>
</machines-archive>

I’m trying to get at machines-archive//item so this ought to would work, right?

<xsl:template match="machines-archive">
<xsl:for-each select=".//item[not(@handle = preceding::item/@handle)]">
<xsl:sort select="@handle" order="ascending"/>
<xsl:value-of select="@handle" /><xsl:text>, </xsl:text>
</xsl:for-each>
</xsl:template>

All the forum posts and mailing list archives I searched assured me that this’d be just fine… Except it wasn’t. It just didn’t work. Operating on all items in the document (//item) generated a distinct, de-duped set as expected, but as soon as I added the ‘.’ to work on a subset (.//item) it didn’t. Eh?!

Head-scratching, template-tweaking and tedious Googling ensued. I tried everything I could think of and after several hours of a lot of searching I eventually stumbled across a solution: EXSLT and set:distinct.

I first needed to add: xmlns:set="http://exslt.org/sets" to my stylesheet’s namespace. Then I could use set:distinct on my items:

<xsl:for-each select="set:distinct(.//item)">
<xsl:sort select="@handle" order="ascending"/>
<xsl:value-of select="@handle" /><xsl:text>, </xsl:text>
</xsl:for-each>

Job done. No one was pointing out this most elegant of solutions anywhere I found on the World Wide Web but it’s easy when you know how…

We’ve just got ourselves a couple of new servers for running SONAR on, for internal use and demonstration. They are quite beefy, with 8GB RAM, 15k disks, and 8 x 3.2GHz threads. We decided to go with centos 5 as the operating system, since we have had pain installing Oracle on our debian platforms. Here is the install procedure I uncovered, which gets the server ready for SONAR on either MySQL or oracle. It covers installing Ruby, RubyGems, lots of useful Gems, Java, and Oracle 10g

yum install -y ruby
yum install -y ruby-devel ruby-docs ruby-ri ruby-irb ruby-devel ruby-rdoc
yum install -y w3m # for text based browsing and downloading, where curl won't work

# download rubygems from rubygems.org with w3m then...
gunzip rubygems-0.9.4.tgz
tar xf rubygems-0.9.4.tar
cd rubygems-0.9.4
ruby setup.rb

# get jdk rpm from java.sun.com and install

# add an /etc/profile.d/java.sh file, containing
##
JAVA_HOME=/usr/java/default
export JAVA_HOME
##
chmod +x /etc/profile.d/java.sh
source /etc/profile.d/java.sh

yum install -y ImageMagick-devel
yum install -y perl-DBI
yum remove -y sendmail
yum install -y postfix
chkconfig postfix on
yum install -y ntp
chkconfig ntpd on
yum install -y mysql-server mysql mysql-devel

chkconfig mysqld on
/usr/bin/mysqladmin -u root password '<password>'
/usr/bin/mysqladmin -u root -h <hostname> password '<password>'

yum install -y gcc
gem install builder
gem install camping
gem install capistrano
gem install cgi_multipart_eof_fix
gem install cheat
gem install chronic
gem install ci_reporter
gem install daemons
gem install fastthread
gem install gem_plugin
gem install hpricot
gem install image_science
gem install jerbil --source http://code.trampolinesystems.com
gem install json
gem install markaby
gem install metaid
gem install mongrel
gem install needle
gem install net-sftp
gem install net-ssh
gem install rake
gem install rmagick  # [ ignore errors about fonts... they are just for the docs ]
yum install -y libxml2-devel
yum install -y libxslt-devel
gem install ruby-xslt
gem install rubyforge
gem install RubyInline
gem install mysql -- --build-flags --with-mysql-config
yum install -y httpd
yum install -y subversion

# install rpmforge for centos 5 as detailed here :
# http://wiki.centos.org/Repositories/RPMForge#head-d766e7aacdee6150c6705d9369aa36c9bcabc139

yum install -y firefox
yum install -y xorg-x11-xauth
yum install -y vnc-server
yum install -y emacs
yum install -y libXtst
yum install -y xorg-x11-utils
yum install -y xorg-x11-resutils
yum install -y xorg-x11-server-utils

yum install -y compat-libgcc-296 compat-libstdc++-296 compat-libstdc++-33 openmotif22
yum groupinstall -y "Development Libraries"
yum groupinstall -y "Development Tools"
yum groupinstall -y "Legacy Software Development"
yum groupinstall -y "Server Configuration Tools"
yum groupinstall -y "X Software Development"
yum groupinstall -y "Administration Tools"

# fire up a vncserver on the remote machine :
vncserver
# and enable connections to the X server from other accounts :
xhost +

# connect an ssh session from your local machine, forwarding the vnc port [ 5900 + screen # ]
# connect your local vnc client to the server
# run firefox on the server
# download oracle 10g from www.oracle.com, thus avoiding having to upload 750M at 512Kb/s
# install as shown here : http://www.dizwell.com/prod/node/681?page=0%2C3
# and add an init.d startup script as described here : http://www.dizwell.com/prod/node/50?page=0%2C4

# done ! reboot to check that your init.d scripts are functioning correctly

I’ve just released a new version of springy, the JRuby-based alternative to Spring’s XML configuration format. It now has a “serialise-to-xml”-feature which allows you to write your whole config to a nicely formatted Spring XML file. Why would you want to do that ? In our case we’ve created a Ruby-based configuration framework modeled after Rails environments which pulls config files from different locations, deals with overrides and so on. The only downside is that this is pretty slow (especially when unit testing), hence the idea of serialising the whole context to XML.

The serialised XML config file is about twice the size of the springy context (~1600 vs. ~800 lines), which shows how expressive Ruby can be. A small example from our configuration:

DAOS = [ :ZoneDAO, :EmailDomainDAO, :DayDAO, :PreferenceDAO,
 :WhatEverDao… ]
DAOS.each do |dao|
bean(dao, “daos.hibernate.#{dao}Hibernate”)
{|b| b.new(“sonarSession”)}
end

This snippet uses metaprogramming to create 20 odd DAOs in 8 lines of code.

Springy has matured since the first release when it was still in the proof-of-concept-stage. We’re using it successfully in Sonar, and there is no way we’ll ever go back to using XML for configuration. Anyway, XML is for girls, according to Hani.

My new favourite feature in JRuby is JavaUtilities.extend_proxy. Basically it allows you to extend an existing Java class with Ruby code. All instances of this class will then have additional Ruby methods on them. This can be handy to fix completely useless APIs like org.w3c.dom.Document. If you’ve ever used Ruby’s xml builder you will know what I mean.

JRuby allows us to do this:

JavaUtilities.
extend_proxy(‘com.sun…internal.dom.DocumentImpl’) {
def tag(sym, *args, &block)
#ruby code omitted
end
}

doc = DocumentImpl.new
doc.tag(“beans”) do
doc.tag(“bean”, “id”=>“mybean”) do
…
end
end
doc.validate!
doc.serialize(System.err)

DOM is finally usable! See serialization.rb for the full code.

I’m a big fan of SVN, and I know it’s in very common use throughout the development community, so I thought I’d give a shout out to an interesting project that extracts some fascinating SVN stats and draws lots of pretty pictures - StatSVN

We’ve now added a StatSVN task to our build scripts, and the figures it kicks out are just great - once you start looking, you can lose yourself in there for ages….

To give some random samples:

• As of three days ago, there were 75130 lines of code in the Sonar trunk, of which Craig has contributed 33.6%, myself 32.1% and Jan 22.2%. As Jan has mostly been working on the Ruby On Rails interface, and Craig and I have mostly been writing the Java backend, what does this say about the relative expressive efficiency and productivity of each language?
• Craig took “Developer Of The Month” for February, with 5286 lines, taking over from me in January with 7546 lines.
• Over 80% of my changes are additions, with under 20% being updates. Craig is more or less the same, with a few percent more modifications - probably due to fixing my occasional “EVIL” quick hacks…
• By far and away the most commits overall get done between 4pm and 5pm, but personally, I do three times as many commits between 2pm and 3pm than any other hour. Probably because I keep saying to myself got to finish this before I go to lunch…
• Jan has not done a single commit before 12 noon However, he’s the only person to have committed between 1am and 2am!
• I do a huge number of commits on a Monday (400+), followed by successively fewer every day until Friday (~60). Does this mean I’m fresher and more enthusiastic after the weekend, or is it that when I start the week, I go for the easiest tasks first, and ramp up towards the more difficult tasks as I go on? On the other hand, overall the most commits are done on a Friday, followed by Tuesday

You can also measure some metrics around files, rather than developers:

• Overall, across all file types, we have an average of around 40 lines of code per file
• 58.7% of the files in the repository are .java, but they contribute 73.3% of the total lines of code
• On average, each .rhtml template is just 26.6 lines of code
• The most amended file in the whole repository is the Rakefile!

I could go on for hours….but it’s nearly 2pm and I need to commit some code, dammit! So install StatSVN on your repository and have a play yourself. It’s fun.