Blog | Archive for July, 2007

A Behavioural Shift in Our Emails

By alistair | Tuesday, July 31st, 2007

Since we started using our own internal Sonar deployment as a test bed, we’ve been noticing a subtle but distinct shift in the way we compose emails. We’re taking just a little bit more care to write informative subjects and supply more context, whether we’re consciously aware of it or not.

One of the main characteristics of the email noise that pervades the enterprise is a lack of persistent context. I long ago lost count of the number of hours of lost on fruitless trawls through my inbox for some nugget of information that I *know* is there, languishing in some long-forgotten recess of my mail client, but I just can’t find it because it’s lost in a morass of meaningless subjects, like “meeting notes”, or “tomorrow”, or – everyone’s favourite – “Re:”. A communication medium such as email, without the situational or emotional context that is generally inferred from ambient knowledge such as a previous verbal conversation followed up with an email, or from the non-verbal clues that provide up to 80% of our meaning, can inevitably lead to serious misunderstandings or just….. noise. A potential goldmine of crucial information – somebody’s inbox – quickly becomes a black hole of knowledge that can never be found again.

What we’re finding since implementing Sonar is that the knowledge that our emails are going to be processed and have themes extracted from them is making us – consciously or subconsciously – put that little bit of extra thought into providing that context in our sentence construction. It only takes a second to change a subject line like “meeting notes” to “notes from production meeting”, or change “tomorrow” to “tomorrow’s development tasks”, but that extra context makes the world of difference when you’re dredging back through your inbox, months down the line.

It also means that the picture of organisational expertise that Sonar can build up can be rich, informative, and ever-evolving. It’s changing the way we communicate, in subtly varied ways, making us provide more relevant information and, frankly, cut out the crap – and that, to my mind, can only be a good thing.

Posted in Organisations & Technology, SONAR | 1 Comment »

Tracing file access on Mac OSX

By Tomasz | Tuesday, July 24th, 2007

I just started working at Trampoline Systems yesterday. Some of you might know my blog already (the one with kitten pics and programming rants). The first thing I did was configuring all the software on my new MacBook. Most of it went all right without any problems, but a few gems didn’t want to install, complaining about jni.h missing. jni.h was on the box, so the problem was basically that the gems were looking for it in a wrong place. Problems like that can usually be solved by strace -e trace=file gem install whatever | grep 'jni.h' and a symlink. Not this time, because Macs does have strace. There’s something called ktrace, but it didn’t seem to provide information I needed.

Full strace may be impossible without heave kernel hacking, but it’s not hard to write a simple file access tracing utility. The one I coded uses LD_PRELOAD-like trick to override open and stat functions in libc. These two usually provide most of the usuful information about file access, and if anything more was needed the utility can be easily extended.

The library is very simple:

#include <unistd.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <dlfcn.h>
#include <errno.h>
#include "errno_names.h"

/* Probably not thread-safe. I have no idea what's thread-interpretation of
dlsym(RTLD_NEXT, ...), even assuming atomic assignment to variables
*/
int (*real_open)(const char *path, int flags, mode_t mode) = 0;
int (*real_stat)(const char *path, struct stat *sb) = 0;

void report_errno(int retval)
{
if(retval == -1 && errno >= 0 && errno <= ELAST) {
fprintf(stderr, " (%s)n", errno_names_table[errno]);
} else {
fprintf(stderr, "n");
}
}

int open(const char *path, int flags, mode_t mode)
{
int retval;

if(real_open == 0)
real_open = dlsym(RTLD_NEXT, "open");
retval = real_open(path, flags, mode);

fprintf(stderr, "open("%s", ...) = %d", path, retval);
report_errno(retval);

return retval;
}

int stat(const char *path, struct stat *sb)
{
int retval;

if(real_stat == 0)
real_stat = dlsym(RTLD_NEXT, "stat");
retval = real_stat(path, sb);

fprintf(stderr, "stat("%s", ...) = %d", path, retval);
report_errno(retval);

return retval;
}

Fist of error names in errno_names.h is generated by Rakefile, which looks like that (stripping the testing-related parts):

desc "Build trace_file_access library"
task :build => "libtfa.dylib"

file "libtfa.dylib" => ["errno_names.h", "libtfa.c"] do
sh "gcc", "-O6", "libtfa.c", "-dynamiclib", "-fPIC", "-o", "libtfa.dylib"
end

file "errno_names.h" do
errnos = []
File.open("/usr/include/sys/errno.h").each{|line|
errnos[$2.to_i] = $1 if line =~ /A#defines+(E[A-Z]+)s+(d+)/
}
File.open("errno_names.h", "w") {|fh|
fh.puts "/*n * This code is automatically generated by the Rakefilen * Do not modify by handn */"
fh.puts "static const char *errno_names_table [] = {"
errnos.each_with_index{|name, i|
fh.puts "  "#{name}", /* #{i} */"
}
fh.puts "};"
}
end

Finally trace_file_access command script sets up the tracing, with the results looking like this: #!/usr/bin/env ruby

ENV["DYLD_FORCE_FLAT_NAMESPACE"] = "1"
ENV["DYLD_INSERT_LIBRARIES"] = "./libtfa.dylib"
exec *ARGV

$ ./trace_file_access ruby -e 'require "rational"'
open("/dev/urandom", ...) = 3
stat("/opt/local/lib/ruby/site_ruby/1.8/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/1.8/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/1.8/i686-darwin8.9.3/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/1.8/i686-darwin8.9.3/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/i686-darwin8.9.3/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/i686-darwin8.9.3/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/1.8/rational.rb", ...) = 0
open("/opt/local/lib/ruby/1.8/rational.rb", ...) = 3
stat("/opt/local/lib/ruby/site_ruby/1.8/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/1.8/i686-darwin8.9.3/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/i686-darwin8.9.3/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/1.8/rational.rb", ...) = 0
open("/opt/local/lib/ruby/1.8/rational.rb", ...) = 3
open("/opt/local/lib/ruby/1.8/rational.rb", ...) = 3
open("/opt/local/lib/ruby/1.8/rational.rb", ...) = 3
open("/opt/local/lib/ruby/1.8/rational.rb", ...) = 3

Posted in Coding | 2 Comments »

Java and Rails integration with GoldSpike

By jan | Friday, July 13th, 2007

While trying to create a unified testing framework (shared between Rails and our Java backend code) I came across ActiveRecordJDBC which is an adapter to use JDBC drivers with JRuby on Rails. It works fine, although it can be a bit complicated to get a DRY database.yml configuration. The goal is to get rid of our dbunit/manually crafted database tests on the Java side by using ActiveRecord fixtures. After some research I found out about the Rails integration project (now called GoldSpike), which tries to make it easy to deploy a Rails app on a Java servlet container such as jetty. As far as I know Thoughtworks uses this approach to deploy their new product, Mingle. GoldSpike is under constant development but it is already usable, although a few patches were required. After everything was set up, a simple


$ rake war:standalone:create
Reading user configuration
Assembling web application
Adding Java library commons-pool-1.3
Adding Java library rails-integration-1.1.1
Adding Java library activation-1.1
Adding Java library mysql-connector-java-5.0.5
Adding Java library bcprov-jdk14-124
Adding Java library jruby-complete-1.0
Adding web application
Adding Ruby gem ActiveRecord-JDBC version 0.4
Creating web archive

creates a .war file which can be directly deployed to a container. I’ve never been a big fan of war files, but in the context of JRuby+Rails it makes perfect sense, because Ruby and all the required gems can be packaged up in a single file. No need to worry about missing gems, C bindings or wrong versions of the installed software, all you need to deploy is Java and jetty (which is pretty lightweight).

Not quite sure what the implications of this are, but it seems like a good alternative to mongrel/CRuby deployments. Performance-wise it looks good, too, though we haven’t done any benchmarking. GoldSpike doesn’t have a project homepage yet, but you can find it on the jruby-extras project page. Let’s wait and see what this will mean for the adoption of Rails in corporate environments (Oracle is currently looking for Rails developers to join the Enterprise 2.0 team, for example).

Posted in Coding | No Comments »

growl-lastfm

By jan | Tuesday, July 10th, 2007

We use last.fm a lot in the office – but one thing I always found annoying was that there’s no easy way to find out what’s currently playing (you need to go to the web page and hit refresh, very distracting) so I knocked up a little ruby script which uses growl to display the currently playing song.


#!/usr/bin/env ruby
require 'rubygems'
require 'ruby-growl'
require 'hpricot'
require 'cgi'
require 'open-uri'

raise "#{$0} <user>" if ARGV.empty?
user = ARGV[0]
already_notified = []

while true do
now_listening = Hpricot(open("http://www.last.fm/user/#{user}"))/".nowListening td.subject"
unless now_listening.empty?
currently_played = now_listening.last.inner_text.strip
unless already_notified.include?(currently_played)
already_notified << currently_played
g = Growl.new("localhost", "ruby-growl", ["ruby-growl Notification"])
g.notify("ruby-growl Notification", "#{user} is listening to",
CGI::unescapeHTML(currently_played))
end
end
sleep 60
end

Make sure that you have hpricot and ruby-growl installed (gem install hpricot ruby-growl -y), then copy and paste the script (or download it) and start it with a last.fm username as parameter. You also need to have “Listen for incoming notifications” and “Allow remote registration” checked in growl’s preferences (System Preferences | Growl | Network).

Posted in Coding | 5 Comments »

Select distinct with XSLT

By mike | Friday, July 6th, 2007

While working on some tweaks to our website I decided that for the archive I needed to be able to select a list of categories used. Should be easy, right? Here’s a sample of the XML:

<machines-archive section="weblog" section-id="3">
<entry id="78" handle="installing-oracle-10g-on-64-bit-centos-5" linked-count="0">
<date year="2007" month="06" date="28" weekday="4">2007-06-28</date>
<time hour="12" minute="35">12:35</time>
<author />
<fields>
<title handle="installing-oracle-10g-on-64-bit-centos-5">installing Oracle 10g on 64 bit centos 5</title>
<categories>
<item handle="ruby">Ruby</item>
<item handle="java">Java</item>
<item handle="programming">Programming</item>
<item handle="code">Code</item>
</categories>
</fields>
<comments count="0" spam="0" />
</entry>
<entry id="75" handle="compulsory-wearing-of-facial-hair" linked-count="0">
<date year="2007" month="06" date="14" weekday="4">2007-06-14</date>
<time hour="14" minute="15">14:15</time>
<author />
<fields>
<title handle="compulsory-wearing-of-facial-hair">compulsory wearing of facial hair</title>
<categories>
<item handle="programming">Programming</item>
</categories>
</fields>
<comments count="0" spam="0" />
</entry>
</machines-archive>

I’m trying to get at machines-archive//item so this ought to would work, right?

<xsl:template match="machines-archive">
<xsl:for-each select=".//item[not(@handle = preceding::item/@handle)]">
<xsl:sort select="@handle" order="ascending"/>
<xsl:value-of select="@handle" /><xsl:text>, </xsl:text>
</xsl:for-each>
</xsl:template>

All the forum posts and mailing list archives I searched assured me that this’d be just fine… Except it wasn’t. It just didn’t work. Operating on all items in the document (//item) generated a distinct, de-duped set as expected, but as soon as I added the ‘.’ to work on a subset (.//item) it didn’t. Eh?!

Head-scratching, template-tweaking and tedious Googling ensued. I tried everything I could think of and after several hours of a lot of searching I eventually stumbled across a solution: EXSLT and set:distinct.

I first needed to add: xmlns:set="http://exslt.org/sets" to my stylesheet’s namespace. Then I could use set:distinct on my items:

<xsl:for-each select="set:distinct(.//item)">
<xsl:sort select="@handle" order="ascending"/>
<xsl:value-of select="@handle" /><xsl:text>, </xsl:text>
</xsl:for-each>

Job done. No one was pointing out this most elegant of solutions anywhere I found on the World Wide Web but it’s easy when you know how…