Trampoline Systems

* Trampoline Description Here

Trampoline Systems

* Trampoline Description Here


Content

Machines

Ideas, thoughts and observations from Trampoline's technical brains

Archive for the ‘Programming’ Category

mccraig

removing global fixtures for ruby tests

By craig mcmillan on July 2nd, 2008

global fixtures are evil, but we’ve got a bunch of unit tests depending on them, so we still need them around

here’s a neat [and generally fast, though a degenerate O(#tables^2) case is possible] way of deleting all fixtures without invoking db dependent ways of ignoring foreign-key constraints, and without loading all the objects into memory :

classes = ActiveRecord::Base.connection.tables.map {|t|
  t.singularize.camelize.constantize rescue nil
}.compact.reject{|cls| !cls.ancestors.include?(ActiveRecord::Base)}

while classes.size > 0
  classes = classes.select{|c|
    begin
      c.delete_all
      false
    rescue
      true
    end
  }.reverse!
end
jan

Open Visualisation Workshop at the Trampery

By Jan Berkel on May 21st, 2008

Trampoline Systems is hosting a visualisation workshop this coming Saturday, 24th May, organised by the Open Knowledge Foundation. Come along if you’re interested in open source visualisation technologies (Prefuse, Flare etc.). The goal is to have a very informal setting to talk about various aspects of visualising data. Find out more in the official announcement.

Hope to see you here!

mike

@media Ajax 2007

By Mike Stenhouse on September 5th, 2007

I have the honour and terror of presenting at @media Ajax on home turf this November. It’s a privilege to be speaking alongside the likes of Brendan Eich (creator of Javascript), Douglas Crockford (inventor of JSON), John Resig (JQuery lead) and about a dozen other top dogs.

In a lineup like that I clearly can’t talk about nuts and bolts Javascript. Instead I’m taking a slightly unusual tack for me: revelations. Since Ajax came along my job has changed in ways I wouldn’t have predicted. Technically I’m a flavour of designer yet after many years of specialising I’ve found myself having to skill up again.

  • To keep a handle on what the rest of the team produce I’ve become a testing fanatic;
  • I’ve had to go back and relearn how to program - not to necessarily produce back-end code but to understand what the real implications of my design decisions are;
  • I’ve been converted to Agile practices as a means of effective collaboration.

None of these things are traditionally within the remit of ‘design’ but they all feed into producing a successful app. To try and describe these changes and what I’ve done about them I will be presenting But I’m a Bloody Designer! on the first day, straight after the keynote by the Ajaxians.

So, the lineup’s great, it’s in London. @media Ajax: coming soon. Say hello if you decide to come…

jan

Springy 0.3 released

By Jan Berkel on August 2nd, 2007

No big changes this time, mainly compatibility fixes for JRuby 1.0. It is now also possible to build the project using Maven, for those too afraid to use rake. Documentation and code for springy are available here.

I’m also happy to announce that Craig Walls, the author of “Spring in Action”, is going to talk about Springy as part of his “Spring Cleaning: Tips for Managing XML Clutter” talk at this year’s No Fluff Just Stuff series of events as well as the Spring Experience 2007 in Florida.

Tomasz

Tracing file access on Mac OSX

By Tomasz Wegrzanowski on July 24th, 2007

I just started working at Trampoline Systems yesterday. Some of you might know my blog already (the one with kitten pics and programming rants). The first thing I did was configuring all the software on my new MacBook. Most of it went all right without any problems, but a few gems didn’t want to install, complaining about jni.h missing. jni.h was on the box, so the problem was basically that the gems were looking for it in a wrong place. Problems like that can usually be solved by strace -e trace=file gem install whatever | grep 'jni.h' and a symlink. Not this time, because Macs does have strace. There’s something called ktrace, but it didn’t seem to provide information I needed.

Full strace may be impossible without heave kernel hacking, but it’s not hard to write a simple file access tracing utility. The one I coded uses LD_PRELOAD-like trick to override open and stat functions in libc. These two usually provide most of the usuful information about file access, and if anything more was needed the utility can be easily extended.

The library is very simple:

#include <unistd.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <dlfcn.h>
#include <errno.h>
#include "errno_names.h"

/* Probably not thread-safe. I have no idea what's thread-interpretation of
dlsym(RTLD_NEXT, ...), even assuming atomic assignment to variables
*/
int (*real_open)(const char *path, int flags, mode_t mode) = 0;
int (*real_stat)(const char *path, struct stat *sb) = 0;

void report_errno(int retval)
{
if(retval == -1 && errno >= 0 && errno <= ELAST) {
fprintf(stderr, " (%s)n", errno_names_table[errno]);
} else {
fprintf(stderr, "n");
}
}

int open(const char *path, int flags, mode_t mode)
{
int retval;

if(real_open == 0)
real_open = dlsym(RTLD_NEXT, "open");
retval = real_open(path, flags, mode);

fprintf(stderr, "open("%s", ...) = %d", path, retval);
report_errno(retval);

return retval;
}

int stat(const char *path, struct stat *sb)
{
int retval;

if(real_stat == 0)
real_stat = dlsym(RTLD_NEXT, "stat");
retval = real_stat(path, sb);

fprintf(stderr, "stat("%s", ...) = %d", path, retval);
report_errno(retval);

return retval;
}

Fist of error names in errno_names.h is generated by Rakefile, which looks like that (stripping the testing-related parts):

desc "Build trace_file_access library"
task :build => "libtfa.dylib"

file "libtfa.dylib" => ["errno_names.h", "libtfa.c"] do
sh "gcc", "-O6", "libtfa.c", "-dynamiclib", "-fPIC", "-o", "libtfa.dylib"
end

file "errno_names.h" do
errnos = []
File.open("/usr/include/sys/errno.h").each{|line|
errnos[$2.to_i] = $1 if line =~ /A#defines+(E[A-Z]+)s+(d+)/
}
File.open("errno_names.h", "w") {|fh|
fh.puts "/*n * This code is automatically generated by the Rakefilen * Do not modify by handn */"
fh.puts "static const char *errno_names_table [] = {"
errnos.each_with_index{|name, i|
fh.puts "  "#{name}", /* #{i} */"
}
fh.puts "};"
}
end

Finally trace_file_access command script sets up the tracing, with the results looking like this: #!/usr/bin/env ruby

ENV["DYLD_FORCE_FLAT_NAMESPACE"] = "1"
ENV["DYLD_INSERT_LIBRARIES"] = "./libtfa.dylib"
exec *ARGV

$ ./trace_file_access ruby -e 'require "rational"'
open("/dev/urandom", ...) = 3
stat("/opt/local/lib/ruby/site_ruby/1.8/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/1.8/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/1.8/i686-darwin8.9.3/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/1.8/i686-darwin8.9.3/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/i686-darwin8.9.3/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/i686-darwin8.9.3/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/rational.bundle", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/1.8/rational.rb", ...) = 0
open("/opt/local/lib/ruby/1.8/rational.rb", ...) = 3
stat("/opt/local/lib/ruby/site_ruby/1.8/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/1.8/i686-darwin8.9.3/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/site_ruby/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/1.8/i686-darwin8.9.3/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/vendor_ruby/rational.rb", ...) = -1 (ENOENT)
stat("/opt/local/lib/ruby/1.8/rational.rb", ...) = 0
open("/opt/local/lib/ruby/1.8/rational.rb", ...) = 3
open("/opt/local/lib/ruby/1.8/rational.rb", ...) = 3
open("/opt/local/lib/ruby/1.8/rational.rb", ...) = 3
open("/opt/local/lib/ruby/1.8/rational.rb", ...) = 3
mccraig

installing Oracle 10g on 64 bit centos 5

By craig mcmillan on June 28th, 2007

We’ve just got ourselves a couple of new servers for running SONAR on, for internal use and demonstration. They are quite beefy, with 8GB RAM, 15k disks, and 8 x 3.2GHz threads. We decided to go with centos 5 as the operating system, since we have had pain installing Oracle on our debian platforms. Here is the install procedure I uncovered, which gets the server ready for SONAR on either MySQL or oracle. It covers installing Ruby, RubyGems, lots of useful Gems, Java, and Oracle 10g

yum install -y ruby
yum install -y ruby-devel ruby-docs ruby-ri ruby-irb ruby-devel ruby-rdoc
yum install -y w3m # for text based browsing and downloading, where curl won't work

# download rubygems from rubygems.org with w3m then...
gunzip rubygems-0.9.4.tgz
tar xf rubygems-0.9.4.tar
cd rubygems-0.9.4
ruby setup.rb

# get jdk rpm from java.sun.com and install

# add an /etc/profile.d/java.sh file, containing
##
JAVA_HOME=/usr/java/default
export JAVA_HOME
##
chmod +x /etc/profile.d/java.sh
source /etc/profile.d/java.sh

yum install -y ImageMagick-devel
yum install -y perl-DBI
yum remove -y sendmail
yum install -y postfix
chkconfig postfix on
yum install -y ntp
chkconfig ntpd on
yum install -y mysql-server mysql mysql-devel

chkconfig mysqld on
/usr/bin/mysqladmin -u root password '<password>'
/usr/bin/mysqladmin -u root -h <hostname> password '<password>'

yum install -y gcc
gem install builder
gem install camping
gem install capistrano
gem install cgi_multipart_eof_fix
gem install cheat
gem install chronic
gem install ci_reporter
gem install daemons
gem install fastthread
gem install gem_plugin
gem install hpricot
gem install image_science
gem install jerbil --source http://code.trampolinesystems.com
gem install json
gem install markaby
gem install metaid
gem install mongrel
gem install needle
gem install net-sftp
gem install net-ssh
gem install rake
gem install rmagick  # [ ignore errors about fonts... they are just for the docs ]
yum install -y libxml2-devel
yum install -y libxslt-devel
gem install ruby-xslt
gem install rubyforge
gem install RubyInline
gem install mysql -- --build-flags --with-mysql-config
yum install -y httpd
yum install -y subversion

# install rpmforge for centos 5 as detailed here :
# http://wiki.centos.org/Repositories/RPMForge#head-d766e7aacdee6150c6705d9369aa36c9bcabc139

yum install -y firefox
yum install -y xorg-x11-xauth
yum install -y vnc-server
yum install -y emacs
yum install -y libXtst
yum install -y xorg-x11-utils
yum install -y xorg-x11-resutils
yum install -y xorg-x11-server-utils

yum install -y compat-libgcc-296 compat-libstdc++-296 compat-libstdc++-33 openmotif22
yum groupinstall -y "Development Libraries"
yum groupinstall -y "Development Tools"
yum groupinstall -y "Legacy Software Development"
yum groupinstall -y "Server Configuration Tools"
yum groupinstall -y "X Software Development"
yum groupinstall -y "Administration Tools"

# fire up a vncserver on the remote machine :
vncserver
# and enable connections to the X server from other accounts :
xhost +

# connect an ssh session from your local machine, forwarding the vnc port [ 5900 + screen # ]
# connect your local vnc client to the server
# run firefox on the server
# download oracle 10g from www.oracle.com, thus avoiding having to upload 750M at 512Kb/s
# install as shown here : http://www.dizwell.com/prod/node/681?page=0%2C3
# and add an init.d startup script as described here : http://www.dizwell.com/prod/node/50?page=0%2C4

# done ! reboot to check that your init.d scripts are functioning correctly
mccraig

compulsory wearing of facial hair

By craig mcmillan on June 14th, 2007

rather splendid picture of our beards and moustaches

further rather splendidly hirsute pictures

mccraig

We want fresh engineer blood

By craig mcmillan on March 20th, 2007

SONAR brings statistical natural language processing and social network analysis to the problem of browsing and filtering large archives. We permit users to see who is talking to who, how much and about what. There are applications in areas including expertise discovery, compliance, forensics, search and alerting.

We are a small company, with large ambition. We number currently 10, and have just secured £3M of venture funding to help us grow our R&D efforts and establish a North American presence.

The SONAR platform consists of a very Ajaxy web front-end in Ruby on Rails, with a Java back-end doing the asynchronous natural language processing and social network analysis.

We are looking for software developers with experience in the following technologies :

Java, Hibernate, Spring framework, JMS, Ruby, Ruby on Rails, Transactions, Transactional messaging, SQL, MySQL, Oracle, Linux, Windows

Experience in all of the above is not a necessity. We are looking for smart people who can learn quickly, love solving difficult problems and can grow with the company. These attributes are far more important than any tick list.

If you are interested, please contact :

Craig McMillan, craig AT trampolinesystems DOT com

Please note : if you are an agency and are not already talking to us, then do not contact us. Really.

alistair

Java vs. Ruby with StatSVN

By Alistair Davidson on February 23rd, 2007

I’m a big fan of SVN, and I know it’s in very common use throughout the development community, so I thought I’d give a shout out to an interesting project that extracts some fascinating SVN stats and draws lots of pretty pictures - StatSVN

We’ve now added a StatSVN task to our build scripts, and the figures it kicks out are just great - once you start looking, you can lose yourself in there for ages….

To give some random samples:

  • As of three days ago, there were 75130 lines of code in the Sonar trunk, of which Craig has contributed 33.6%, myself 32.1% and Jan 22.2%. As Jan has mostly been working on the Ruby On Rails interface, and Craig and I have mostly been writing the Java backend, what does this say about the relative expressive efficiency and productivity of each language?
  • Craig took “Developer Of The Month” for February, with 5286 lines, taking over from me in January with 7546 lines.
  • Over 80% of my changes are additions, with under 20% being updates. Craig is more or less the same, with a few percent more modifications - probably due to fixing my occasional “EVIL” quick hacks…
  • By far and away the most commits overall get done between 4pm and 5pm, but personally, I do three times as many commits between 2pm and 3pm than any other hour. Probably because I keep saying to myself got to finish this before I go to lunch…
  • Jan has not done a single commit before 12 noon :-) However, he’s the only person to have committed between 1am and 2am!
  • I do a huge number of commits on a Monday (400+), followed by successively fewer every day until Friday (~60). Does this mean I’m fresher and more enthusiastic after the weekend, or is it that when I start the week, I go for the easiest tasks first, and ramp up towards the more difficult tasks as I go on? On the other hand, overall the most commits are done on a Friday, followed by Tuesday

You can also measure some metrics around files, rather than developers:

  • Overall, across all file types, we have an average of around 40 lines of code per file
  • 58.7% of the files in the repository are .java, but they contribute 73.3% of the total lines of code
  • On average, each .rhtml template is just 26.6 lines of code
  • The most amended file in the whole repository is the Rakefile!

I could go on for hours….but it’s nearly 2pm and I need to commit some code, dammit! So install StatSVN on your repository and have a play yourself. It’s fun.

mccraig

TestNG : open source in action

By craig mcmillan on January 23rd, 2007

i came across a problem with our java unit testing package, TestNG : our IHookable implementation, which sets up transaction contexts for test methods, wasn’t being given details of Exceptions which occurred during the running of the test methods

downloading the source for TestNG, it turned out to be an easy fix, a one-liner. i submitted a patch to cedric, and a couple of hours later it was accepted and checked in

if only all open source experiences were so smooth