Blog | Archive for January, 2009

java signed types fail

By jan | Thursday, January 29th, 2009

Ok, another fail post, this time regarding java.

Remember, Java has only signed types (int, byte, short, long) – char is the only exception.

WTF?

James Gosling says (source):

Quiz any C developer about unsigned, and pretty soon you discover that almost no C developers actually understand what goes on with unsigned, what unsigned arithmetic is.
Things like that made C complex. The language part of Java is, I think, pretty simple.

Ok, so he is basically saying that J. Random Developer is too stupid to care about differences in signedness, and decided to make them all signed, maybe for consistency reasons or who knows. Sometimes you don’t care about signed variables. Say for example you’re dealing with any sort of decoding/encoding problems (like network protocols). Now you have to make sure that you don’t trip over sign problems (esp. with implicit casts). Here’s a snippet from the JRuby codebase (base64 decoding routines):

   private static byte safeGet(ByteBuffer encode) {
     return encode.hasRemaining() ? encode.get() : 0;
   }
   ...
   int s = safeGet(encode);
   while (((a = b64_xtable[s]) == -1) && encode.hasRemaining()) {
      // do something
   }

In Java, a byte can have a value from -128 to 127. This code above works fine, except when you feed it data which is not just in the range of displayable ascii characters, because the variable “s” will then potentially be negative and cause an ArrayOutOfBoundsException. So the general recommendation is to use the next bigger signed type (short instead of byte, long instead of int). If you’re using someone else’s API (java.nio.ByteBuffer in this case) you don’t have the choice and therefore you have to be extra careful when using it. In this case the right thing to do is a bitwise AND with 0xFF to strip the signed part of the int.

  while (((a = b64_xtable[s & 0xff]) == -1) && encode.hasRemaining()) { // do something }

Here’s the JRuby bug report and a general introduction to java sign issues by Sean R. Owens.


Porting Pearsons to Postgres. Performance?

By david | Thursday, January 29th, 2009

I’ve uploaded a version of the Pearson’s Coefficient code which runs on postgresql. You can download it here.I wrote this as an experiment to see if Postgres could help us with some of our MySQL performance woes.

Some brief experimentation suggests that once you fix PostgreSQL’s ridiculous default configuration the performance story is relatively happy. At small sizes MySQL is moderately faster, but as the sizes get large PostgreSQL seems to take the lead. I don’t have any sort of formal benchmark yet: This needs much more testing before I can definitively claim either is faster than the other, but for now the signs in favour of PostgreSQL are promising.


Has Many + non default primary key loads incorrect data in Rails 2.2.2

By emma | Thursday, January 29th, 2009

I found an interesting bug in Rails 2.2.2 yesterday. I couldn’t find a similar bug on the rails lighthouse so created a new ticket. What was most interesting though, was how quick the rails core team picked up the bug and assigned it to someone.

It turns out that the bug had already been fixed in the current master branch of the rails git repo, though apparently no one had noticed it’s existence because I can’t find any references to this anywhere. I guess the fix in activerecord, which is almost identical to my fix below, will form part of the next release whenever that is.

I assume this is probably also the case for other has_* relationships, but have not verified.

I have a has_many association from class Foo to class Bar, where, for this specific relationship, the primary key on Foo is not id, nor is the foreign key on Bar id.

class Foo
  has_many, :bars, :primary_key => 'a_non_standard_key_name', :foreign_key => 'another_non_standard_key_name'
end

The relationship is one way, I have no need to navigate from Bar back to Foo, but only call a_foo.bars.

This works fine when working with a single object, but breaks down when you want to do eager association preloading to avoid n+1 query problem of loading bars for many foos.

When performing the following you find that

f = Foo.find :all, :include => :Bar
f.bars = [SOMETHING_UNEXPECTED]

The reason is that ActiveRecord creates the preloading query based on the default primary key of Foo (normally id).

It queries for Bar.another_non_standard_key_name matching Foo.id not Foo.a_non_standard_key_name

This causes seriously unexpected behaviour, and could easily go unnoticed since no errors are thrown.

I have found the hook in ActiveRecord where this functionality should be included and monkey patched for my system, because I need it now. I can’t vouch for it’s correctness, but we have many many specs for our product and none of them have broken because of this.

I’m running frozen rails 2.2.2

vendor/activerecord/lib/active_record/association_preload.rb, line 221

Change

primary_key_name = reflection.through_reflection_primary_key_name

to

primary_key_name = reflection.through_reflection_primary_key_name || reflection.options[:primary_key]

Hope this helps someone!


Yet another MySQL Fail

By david | Monday, January 26th, 2009

mysql> create table stuff (name varchar(32));
Query OK, 0 rows affected (0.24 sec)

mysql> insert into stuff values (‘foo’), (’1′), (’0′);
Query OK, 3 rows affected (0.00 sec)
Records: 3 Duplicates: 0 Warnings: 0

mysql> select * from stuff;
+——+
| name |
+——+
| foo |
| 1 |
| 0 |
+——+
3 rows in set (0.00 sec)

mysql> delete from stuff where name = 0;
Query OK, 2 rows affected (0.09 sec)

mysql> select * from stuff;
+——+
| name |
+——+
| 1 |
+——+
1 row in set (0.00 sec)

mysql> create table stuff (name varchar(32));
Query OK, 0 rows affected (0.24 sec)

mysql> insert into stuff values (‘foo’), (’1′), (’0′);
Query OK, 3 rows affected (0.00 sec)
Records: 3 Duplicates: 0 Warnings: 0

mysql> select * from stuff;
+——+
| name |
+——+
| foo |
| 1 |
| 0 |
+——+
3 rows in set (0.00 sec)

mysql> delete from stuff where name = 0;
Query OK, 2 rows affected (0.09 sec)

mysql> select * from stuff;
+——+
| name |
+——+
| 1 |
+——+
1 row in set (0.00 sec)

mysql> WTF????
-> ;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘WTF????’ at line 1

So, what’s going on here? I said to delete everything where the name was 0, but it deleted the row ‘foo’.

The following might help:

mysql> create table more_stuff(id int);
Query OK, 0 rows affected (0.19 sec)

mysql> insert into more_stuff values(‘foo’);
Query OK, 1 row affected, 1 warning (0.00 sec)

mysql> select * from more_stuff;
+——+
| id |
+——+
| 0 |
+——+
1 row in set (0.00 sec)

When you try to use a string as an integer in MySQL, it takes non numeric strings and turns them into zero. So when you test name = 0, it converts name into an integer and turns that into 0. Consequently strings which can’t be parsed as an integer result in true for this test.

At this point I would rant about how mindbogglingly stupid this behaviour is, but I don’t think I can really be bothered.


JRuby + Clojure’s Immutable Data Structures = Easy to maintain, application data-model.

By Daniel Kwiecinski | Thursday, January 22nd, 2009

Implementing an application with rich data-model which can be updated by multiple UI controls, many concurrent threads with undo/redo functionality may be somewhat cumbersome. In order to ease this task, the functional programming paradigm with the immutable data structures turned out to be useful.

Because all good developers are lazy, one should seek for reuse rather than reinventing required tools, especially when there is good existing one. I tried to follow that path. Since we are using JRuby as our language of choice here at Trampoline, I decided to look more closely at clojure’s immutable data structures. It is straightforward to use Java classes from JRuby which is described in many places on the web already (here, here & here). The unknown to me was how can I use clojure’s objects from Jruby. Apparently clojure data structures are delivered as pre-compiled java classes and no runtime interpretation/compilation of clojure scripts is needed. The task turned out to be very easy.

The simple implementation of graph data structure with no deletion functionality looks as simple as:

basic_graph1
In order to have Clojure collections look more like Ruby ones one can define aliases for their methods:

persistent_map

Unfortunately (or fortunately due to different contract) we can not do it with all the methods. Particularly with mutating ones. That’s because Ruby’s = (assign operator) semantics is to return the value being assign. It is analogous to []= method as well. So even if we redefine the []=(key, val) method so that the method returns the updated version of the collection, the Ruby interpreter will step into the scene and wrap the whole method, so that it eventually returns val. Anyway, whether this is good or bad is the topic for a whole other post.


“Red Herring 100 Global” Awarded to Trampoline

By Trampoline Systems | Tuesday, January 20th, 2009

Trampoline Systems today announced that it has been awarded as one of the Red Herring Top 100 Global Companies. Red Herring Top 100 Global Companies are chosen from winners and finalists of the previous North America, Europe and Asia Red Herring Top 100 Companies. Winning and finalist companies from the previous three years were eligible for this outstanding award. Out of 1,800 successful and highly eligible companies, the Red Herring editorial team deployed a detailed process to drill-down the best companies first to 200 finalists, then to the top 100 winners of this global award. Evaluations were made on both quantitative and qualitative criteria, such as financial performance, innovation, management, global strategy, and ecosystem integration. The announcement of the winners was made at the Red Herring 100 Global taking place in San Diego from January 14-16, 2009. Present among the finalists were elite executives and venture capital leaders from around the world.

“We were so pleased to announce Trampoline as a Red Herring Top 100 Global Company,” commented Red Herring publisher Alex Vieux. “Trampoline has proven to be a company excelling in their industry and its ripples have turned into waves. It was difficult for us to narrow down, but we are pleased to have included Trampoline Systems in our list of promising companies. We look forward to the changes it make to its industry in the future.”


mysql cast to floating point

By mccraig | Wednesday, January 14th, 2009

discovered another mysql trick

if you are experiencing underflow with mysql fixed point arithmetic, you may need to force a floating point evaluation. cast() does not support cast to floating point so multiply by 1e0 instead

e.g.

select 1000000000000000000000000000000 * ( 1 / ( 1000000000000000000000000000000 ));

returns the wrong answer, whereas :

select 1000000000000000000000000000000 * ( 1 / ( 1e0 * 1000000000000000000000000000000 ));

is fine and dandy


Flow of control in Debian maintainer scripts

By jan | Wednesday, January 7th, 2009

The Debian package installation process (as described in the Debian policy) is fairly complicated, at least internally. During the process of building a Debian package for our software I often had to check the policy manual for the order the various maintainer scripts (e.g. postinst, prerm etc.) are called. To complicate things further, both the old and new scripts get called (at least during an upgrade). There are various “error-unwinds” (= rollbacks) and final error states.

It struck me that a visual representation would make things a lot easier, so here’s one I knocked together in OmniGraffle, for future reference:

maintainer scripts call sequence

maintainer scripts call sequence


New York Office
234 5th Avenue, 4th Floor
New York, NY
10001, USA

London Office
The Trampery
8-15 Dereham Place
London EC2A 3HJ
United Kingdom