Ruby, Web Applications

Haml: Round Deux

So, with a comment on an earlier post about Haml, saying that it had had a major update I decided to give it another go. This time in conjunction with the web framework Ramaze.

While Haml has definitely improved but there was a definite sensation of returning to an abusive relationship. Haml is very cool but it can also be a colossol bitch to use. Mostly because of its whitespace lovin’ ways. Care to punt as to what the root cause of this issue was:

Illegal nesting: content can’t be both given on the same line as %form and nested within it.

I’d put a space character between my attribute curly braces and the form tag declaration:

%form {:action => 'url'}

when I should have put:

%form{:action => 'url'}

Do I really need this level grief just to avoid writing closing tags?

Haml does, as its advocates say, bring a clean and clear syntax to the table but it also brings a bundle of its own issues along the way. Particularly time consuming is having to adjust indents when placing content at different layers. The strict two spaces rule means you have to adjust blocks and even if you set up your browser to do soft tabs at two spaces (just for one syntax) it is still easier to play around with markup than Haml.

One feature I did like a lot since last time is that variable interpolation is a lot easier now:

%p== My variable goes #{@here}

Inline tags are still hopeless but that’s true of mostly of the lightweight templating systems.

Standard
Ruby

Still waiting for a Bulk Gems update…

I need to post about this again because life was just getting impossible on my cheapo virtual box. Well, I say cheap but it probably still has 75Mb of free memory that was just being hosed by RubyGems.

I tried updating to Ruby 1.8.7 but that had no effect because the culprit is really Gem itself.

The final solution (after naturally getting off 1.8.6, do it today…) turned out to be building and installing Gem from the 2.0 source tarball. Since getting away from 1.1.1 my Gem experience has been a lot smoother. Ramaze and the monsterous Merb installed smartly and without fuss. Don’t suffer for longer than you have to!

Standard
Programming, Ruby

Synthesis: decreasing tests, increasing reliability

George Malamidis gave a talk at Ruby Tuesday last week about Synthesis which is a kind of mock verification framework. It was the second time I have heard George talk about Synthesis but I really didn’t get it first time (partly due to a pointless debate at the first talk about functional testing (interesting but irrelevant)). A different environment and it just zinged into place for me.

Synthesis ensures that if you mock an interaction between two collaborators you also have a non-mocked test for the method calls in the collaboration. In short it shows when you have stopped unit testing and started testing your mock framework.

Like most people at the talk I could see immediately how that would lead to less worrying about mocking and the good news is that Java port is being worked on.

I wonder if the issue will arise in the Geek Night on Wednesday?

Standard
Programming, Ruby, Scripting, Web Applications

Sinatra and Haml

On Monday I was meant to be learning how to use the Grails framework (following up the impressive Grails site that Sky has launched) but I instead got distracted into scratching another itch, using Sinatra with JRuby. Since I was doing a little REST application I thought I would also give Haml a go as it promised a far easier way of generating HTML responses.

Installing both Gems was easy as ever and Sinatra was really easy to understand conceptually. Put the HTTP method name, the URI pattern you want to match and then the return value of the block is what goes back to the client in the response.

So for example to map “/hello” to the plain text response “Hello World” you simply have.

get "/hello" do
  "Hello World"
end

Fire up the script (with necessary requires) and a HTTP server is set up and running. It really couldn’t be simpler. You can use a :symbol in the URI matcher and then access it through the params hash. Within half an hour I was starting to add resources into my served HTML and I felt like master of the REST UNIVERSE.

Sinatra is a really smart piece of code that makes it simple to write a basic web application. I have loads of ideas of how it could be useful but one of my first thoughts was that it actually does a good job of solving the issue of GUI platforms.

My next project is to see how it works with posting data back to the app but on the face of it it all seems straight-forward.

Haml is another story. It is undoubtedly a good idea and on the right track. A DSL for creating HTML the method html for example creates a block html tag  while p “Hello World” creates a paragraph tag with the parameter as the content. Hash parameters sent to the method become attributes of the class.

It is much quicker than generating the HTML by hand but often not by much. That is because Haml has not found the same easy metaphor for code and content that Sinatra has. It is very picky, constantly harping about two-space indentation (no more, no less, don’t dare use tabs you bastardo!) for example, it sometimes throws a stack trace that failed to make it to Sinatra leading to a blank screen on the browser and hunt into the console for a clue as to what has gone wrong.

The documentation is to poor to figure out why the library doesn’t accept what you are doing and I am still baffled as to whether the module handles inline tags or whether you are meant to devolve all of that to Textile. Passing parameters to Haml templates seems unnecessarily complex (the only way I could get it to work is with normal string interpolation, it’s a solution but it seems to break the DSL concept) and mixing inline formatting with parameters baffles me still.

Haml makes more sense to me than RHTML style templating but its claim to simplicity and enjoyment seems to come only with a deep understanding and long experience with using it. It took me 20 minutes to fall in love with Sinatra but it took 20 minutes for me to get a working Haml page. And I didn’t enjoy myself doing it.

Standard
Programming, Ruby, Scripting

CouchDB

What is CouchDB?

CouchDB is a dedicated document based database that kind of puts it in the same space as Exist, Xindice and Oracle Berkeley XML Db. What makes it very different is that instead of building around XML it is built around Javascript. The document storage format is JSON and the query language is Javascript. Something like Exist uses a minimum of well-formed XML for data and XPath or XQuery for querying.

Now the merits and flaws of the various markup languages is really holy war and I don’t want to get into it too much. I think it is enough to say that CouchDB aims to be lightweight in implementation without being lightweight in features or performance. JSON is very popular in the web space and by focussing on making common cases easier it is less work to use than XML. The more complex the data or specification requirements though the less viable it is as a solution.

Using JSON means that stored data is extremely flexible in structure, unlike relational data you can have very gappy or bitty information and not have a problem. Take something like a customer relationship management system. This is often a poor fit to relational data because you tend to discover information in small tranches. Initially a lead might be a first name and a telephone number. As the relationship develops you add a surname, a company name, a position, an collection of issues or queries and so on. JSON is a really good way of capturing this evolving picture of things.

Using Javascript to query this data surprised me initially, I feel pretty comfortable with my limited Javascript skills and therefore didn’t have a problem with it as a choice. I’ll come back to this later in the post but actually the idea sounds strange but is a natural fit when you use it. Again it is about the right tool for the right job but in the context of the data you are using in CouchDB creating a query by combing iterators, if statements and the functional programming map function is a better fit than trying to adapt SQL.

The final component of CouchDb is the functional language Erlang which is used to service the HTTP REST interface that CouchDB uses to provide an interface to the database. Erlang provides very cheap, lightweight concurrency that provides a good fit for handling lots of HTTP requests.

The interesting thing is that CouchDB automatically provides the REST front end that you tend to build on top of Java object stores like JBoss Cache or Coherence (which I have also been looking at recently). With those products you tend to stay native until you need to share data with other languages or systems and then you tend to REST serve it out as XML with a lightweight HTTP server. If you can see that need ahead of time then CouchDB might well be compelling due to the built-in support.

Installing

After reading about CouchDB I wanted to take a look at this new document database in a bit more detail. I managed to install the base tarball from the Google Code page easily. I then cut and pasted the Ruby sample REST code from the CouchDb Wiki into irb and on running the server I found that I could store and retrieve documents really easily. The combination of interactive Ruby and the REST interface made it easy to play around with data objects.

However when trying to run the tests via the administration web page only four of the tests would pass. The only error message was just an Error Code of 136. Since I seemed able to store and pull documents (and could confirm that the datafile had been generated and had data) I put it down to alpha flakiness and pushed on.

It was only when I tried executing ad-hoc queries and kept having them fail that I realised what was linking my problems. Every time the main Erlang server handed off a query to the Javascript interpreter the process died. Erlang is really resilient to thread death so there was zero impact from this.

CouchDB uses the Mozilla Spidermonkey libraries to construct its own View Server program which is configured via the couch.ini file. Having double-checked that the interpreter was there, was executable and could read the main.js script it was passed I was baffled. I decided to pull down the code base from SVN and then rebuild it. Building from SVN is a bit more involved than the tarfile and its worth double-checking the documentation on the wiki before you get stuck in. Having built and installed myself a new copy I still had the same problem and was feeling pretty cheesed off. With no other source of help I headed to IRC and checked out the Freenode #couchdb channel. There Christopher Lenz gave me a good steer that he had had a problem with Erlang’s HIPE (high performance virtual machine) on Debian.

I had been using Erlang 5.5.5 which I think I used apt-get to obtain. Downloading the latest Erlang release (5.6.1, they release pretty frequently) I tried building that from source and hung when configure tried to check for floating-point exceptions. Running configure with disable-hipe allowed configure to complete and then Erlang was straight-forward to build and install.

Restarting the CouchDB server I found my queries were working and all but one test (“conflicts” fails on an error assertion and this is apparantly a known issue) passing, hurrah!

The machine I had been working on was a Feisty Fawn Ubuntu instance so I loaded up Gutsy Gibbon in VM Ware Player and tried building Erlang and CouchDb there. Erlang was fine, even with Hipe. For CouchDB I used the latest Erlang I had built and used apt-get to obtain the Spidermonkey and ICU depdencies. Everything worked out of the box with this combination.

This problem highlights a few of CouchDB’s weaknesses. Firstly it is all bleeding edge stuff, Erlang is bounding along and so is CouchDB. Secondly there is no test suite in the source distribution that can help troubleshoot issues. Finally the error logging is tricky to understand for a neonate. I’m certain that having an EUnit test suite would have directed me towards the fact that the Javascript was failing much quicker than using trial and error and IRC.

The final issue was where CouchDB stores its logs and datafiles by default. These default to /usr/local/var which is not really where I want to store data like this. /var is a more natural choice and FHS seems to agree. You can change this when building in the etc/couch.ini.tpl file of the source directory but it would be nicer to have a more natural choice by default.

I also tried to install CouchDB on OSX without using a packaging mechanism but instead referring to an instance of Spidermonkey 1.7 I had on my system. While I could get configure to accept the Javascript library it wouldn’t recognise the jsapi.h header, maybe because configure doesn’t define the right Macros when it tries to build the test file.

Working with CouchDB

My first experience of CouchDB’s web interface was that it was good-looking but that the error messages, unresponsive links, Firebug warnings and occasional Javascript pop-ups were all signs that it was a work in progress. On fixing my passthrough to Javascript I had a very different experience of a slick and good looking interface that uses amazingly responsive AJAX to provide a really good experience.

Learning how to generate queries for example is easy with the in-built query browser (I’ll save the actual syntax for a later post) as the feedback from the system is very quick. Similarly creating new databases, documents and fields is actually not torturous but slick and quick.

I’m certain this is because Erlang and AJAX form a natural partnership in creating and servicing small requests that need to be handled quickly. I may be wrong but this is certainly my most positive AJAX webservice experience to date.

The CouchDB wiki contains information on getting started with various languages and I choose Ruby. It was a simple cut and paste into irb and then it was straight-forward to interact with database from the shell and the web client.

One of CouchDB’s more unusual feature seems to stem from Erlang’s concurrency model. Data is assigned a revision and in addition to applying multiple changes in order you can also view previous revisions. That’s a pretty weird feature compared to most of the current datastores. You also need to refer to a revision if you want to update a record. If the revision of the target of the update has changed when you submit your changes your changes get refused. The revision mechanism is also the basis of the replication mechanism although there isn’t enough documentation to understand when the replication pairs check their revision ids.

Incremental data-construction is easy via the web-gui but you need to fiddle a little bit to get the revision number to target if doing it programatically. Presumably a library would make that easier going. For information that has little inherent structure or is very gappy or incremental then CouchDB is a great data store and is currently occupying a niche as far as I know.

Standard
Ruby, Scripting

Dumb Ruby Gems mistake

So on and off over the last week I have been trying to get RedCloth to work in both Ruby and JRuby. Despite getting the gem I kept getting a failure when I tried to require ‘redcloth’. When I finally got Textile support working in Java before I had managed to weave the two or three lines of Textile magic the tutorials promised I shrugged and took the task off my GTD list.

Except that today my stubborn side told me to take a look at the Ruby Gems documentation after seeing an answer to a similar problem on a mailing list. Sure enough there in Chapter 3 was the explanation that you need to require Ruby Gems before the actual Gem you want to use.

Heading to irb and sure enough:

require 'rubygems'
require 'redcloth'

Does the trick and I am finally appreciating the library. Now maybe I’m slow or perhaps the same code on the RedCloth page should have included a full script that would regardless of what was in your RUBYOPT environment variable.

Standard
Java, Ruby, Web Applications, Work

Working with Rails

I have recently struggled to try and get a prototype web application done with Ruby on Rails. It has been a really great experience and like all such experiences consists of both good and bad moments. Things that are particularly striking coming from a Java Web environment:

  • Integrated console
  • No recompile/build/deploy cycle
  • No mix and match components
  • No real programming language skill required.

It is also remarkable how quickly people revert to the level of thought, planning and execution that you might use for a shell script. Even I have found myself getting caught up in “playing” with the app interactively rather than being focussed on creating behaviour and functionality in a structured way.

Elaborating on these points; in Java web development the first issue is usually building up your component stack. Some people just choose Spring  but those people are idjits! If you are working purely in a web tier without any need for any backend interaction then it is probably a good bet but generally you want to put some thought into how you are going to assemble your various stacks. You tend to have to choose your MVC web framework, your persistence framework and your service framework (if appropriate). You often have to give some thought to your caching and messaging frameworks if they are relevant.

With Rails you just use the appropriate Active Module or Rails built-in features. If there is something that is going to ease some pain for you then it is going be operating as a plugin. That’s it, no framework holy war. It also means that there are a lot of applications that are just not going to be suitable for a Rails application as all those Java components have various strengths for different environments. I certainly wouldn’t want to tackle a legacy database with ActiveRecord for example. Not that it couldn’t be done but I wouldn’t want to do it.

The zero-turnaround is impressive after the build-deploy-check cycle. It was a wow factor last year when I saw the Phobos framework being demonstrated by Sun and its not lost any of its shine. By separating its deployment environments so ruthlessly Rails is able to deliver a really positive developer experience.

The interactive console takes a bit of getting used to in terms of faking browser requests but once you get used to doing so it is another tool that you wonder how you’ve managed without for so long. It’s much more intuitive to use than setting up a remote Java debugging instance (although admittedly you do similar tasks in both). It allows you to scratch those itchy “why?” questions.

And finally that lack of programming skill? A bit controversial perhaps? Well I feel that in recent weeks what I have been learning is how to manipulate Rails. The fact that it is implemented on Ruby may allow a lot of features to be implemented in the way they that they are but you are very rarely called upon to show Ruby madskillz. Instead the majority of the time you are simply plugging little customisations into the Rails framework. It is called Rails for a reason after all, when you are on them you are amazingly productive but if you can’t package your problem into a Rails solution then you are out of luck, you’re going to have to develop your own solution and that is going to be hard work. I have been trying to develop my Ruby skills (that is a whole other story) but the truth is you can bang out something that is acceptable with very little Ruby knowledge. You can go a lot of the way purely by mastering Ruby’s hash syntax.

So have I been converted? Well for all its problems I do have a hankering to get back to my great big Java applications with their holy wars and heavyweight processing. After all being on the Rails is fine for getting things done quickly but it can feel claustrophobic. I am also really glad that after getting distracted by the whole JSF controversy the challenge Rails presents to the status quo of web development means that a lot of the Java frameworks are starting to respond to the real problems faced in web development and ensuring that the easy stuff should be easy.

Standard