Thursday, June 6, 2013

Pulling on a thread - amatch, rice, rdoc

Ever have a sweater with a loose thread?  You pull on it and the next thing you know you've got a ball of yarn and no sweater.

My loose thread starts with trying to write a Ruby-on-Rails application.  It seemed so straightforward.  Along the way, I needed do approximate text-matching so I Google'd and found the amatch gem.  Installed it, hooray!

But not so fast.  I was doing search using approximate match and Amatch::Sellers#search returns a score but doesn't tell you where it found the string.

So I downloaded the source code, figuring "how hard could it be to add the position"?  Let me tell you a couple of ways:

  • The code is a few years old and the Ruby infrastructure has changed since it was written.  The lesson here is to get a clean run of all rake tasks before you hack the code.
    • Unit tesing is now done via Minitest, for example.
  • The C-code is in one large file.  No the original author wasn't being silly, RDoc couldn't handle cross file references at the time.
  • Macros and lots of them.  Not my cup of tea.
The macro thing was the deal-breaker on this piece of source.  The Levenshtein and Sellers algorithm's are calculated using dynamic programming.  In order to figure out the search position, you have to navigate a cost matrix looking for binding constraints.  The macro-based version didn't retain the calculated cost matrix so I couldn't backtrace it to find the position.

So I ported the code to Rice/C++ and split it into multiple small files.  Rice was pretty easy to use and handled all of the type juggling between Ruby and C++.

I got the code working (the original author had a decent test suite) so refactoring of this sort went pretty well, maybe 8-10 hours.

Then it was time to produce the documentation.  Queue exploding sounds.  RDoc doesn't understand Rice/C++!

No comments:

Post a Comment