Python, Web Applications

Django in 24 hours

Last Saturday afternoon I decided to learn Django. It was 2pm on the first day of SiCamp 2008 in London and being the only developer in the room at that point I decided that I should do whatever I felt would be the best option to get an application running by 2pm the next day.

Previously I have done some Google App Engine and the experience convinced me to give Django a go after I found myself, by intuition, creating a GAE project structure of (views), and a directory called templates that contained templates. I was then disappointed to find the whole world had got there before me.

So, Django in 24 hours, baptism of fire. What do I think now looking back on the experience?

Everyone has told me that the Django documentation is good and I think I have to concur. Not everything is so clear that when you’re speed reading in one window and typing in another it works first time but importantly nothing in the documentation is actually wrong. When stuff is not working a second, careful look at the documentation got me back on track.

Importantly Django’s core model of web development is sound and intuitive. My editor had around ten files open for the project and the flow of adding something to the application did naturally flow from url to handler to view to model. Maybe the only quibble I have is that the file is deceptively named in MVC terms.

The core of the framework is amazingly concise, I spent the majority of my time thinking about the problem not about the framework API. Binding a URL to function made sense, having to specify a template instead of having one inferred from the method name was maybe my one criticism in the method handler but on the plus side it does allow for flexibility in handling requests. Handing off from one request handler to another was very easy.

Django templates are both amazing and annoying. The syntax and principles are amazing, it was easy to play around with the pages and the template inheritance was really powerful for avoiding duplication. However when I transferred the application from self-serving to mod_python the template generation was very wobbly when compiling changes from the file system. Of course this could also have been mod_python but it was the latest 3 series stable source compiled for the machine. I’ve used Jinja2 previously when generating HTML in Python and might be tempted to stick with it in future.

Django models are great, I hate ORM but I really liked syntax for defining persistence properties and I liked the way that you don’t have the fact that you are really dealing with SQL hidden away from you. It genuinely seemed a more convenient way of expressing the data model rather than an OO wallpaper over relational data storage. I didn’t feel the need to add domain logic to the models but I felt like it wasn’t really polluting the model to do that either.

One thing that didn’t work at all was changing the relations between models; it took me two or three attempts to finally model the relationships between the data concepts. Each time I changed a Foreign Key or Many to Many relationship I ended up deleting the database (SQLite3) as I couldn’t figure how to migrate from the old schema to the new.

One reason for choosing Django was the idea that I wouldn’t have to write the backend code as the admin stuff would be right there for me. It took me a while to get the 1.0 admin to fire up but once it was running it did perform as advertised. One of the attractive things about the application was that the data model followed the conceptual language of the solution in a really powerful way. You could use the admin interface to have a Devotee perform Devotion to an Action. My geek excitement peaked anyway, YMMV.

So them’s the highlights of the experience. Overall Django delivered me a rapid web development process in an intuitive, powerful way and lived up to nearly all of the claims made on the tin. Deploying to Apache/mod_python was painful but most of the pain surrounded the infrastructure of my box (multiple versions of Python, Apache config files) and my lack of mad Apache admin skillz.

I would happily tackle another project in it again.

Perhaps of interest is how the Django development experience matches up against Rails or GAE ¬†which would have been the other obvious choices. GAE would have been very similar but the deployment would have been better and I wouldn’t have had any automatic admin. In retrospect it may have been a better choice for a hack party type event. It certainly would have been my choice for personal projects for easy of deployment but now I have one Django app running perhaps that isn’t as relevant any more. Certainly the thing that has kept me from GAE before, the pain of data migration, doesn’t seem that better in Django (except that you control the datastore and its contents).

Compared to Rails?

  • Admin is much more awesome than scaffolding.
  • Django ORM is much less complex than Active Record, all the data required to create, deploy and use the object is in one place. Django doesn’t have Migrations but has its own brand of database versioning pain.
  • RSpec is awesome (despite its monkey patching of Object) so you aren’t going to beat Rails for easy testing.
  • Django templates are more powerful and easier to use than Erb but you have a lot of Ruby templating options so it’s hard to make a complete comparision. They probably both similar in the sense that you can find a template library that suits your preferences. Django is the purely solution out of the box.
  • Routing and Controllers are much less involved in Django than Rails.
  • Django is less opinionated about how you structure your application directories, which I like.
  • Django doesn’t bake-in AJAX components but is “batteries included”, Rails probably generates better Web2.0 style apps for less effort.
  • Finally Django uses only a few code generators because its basic structure is far less involved. It also generates far less “stuff” for each MVC element which I quite like as I don’t tend to use everything Rails generates.

Okay detailed analysis over, what’s the high-level view? Django and Rails are similar experiences but I think the major differences between them are almost what you could say about Python and Ruby. In Django you are going to get simplicity, clarity and a real choice of how you plug your infrastructure components together. In Rails you are going to get magic up front which is cool but also magic at the back end, which is not cool. Ultimately I think the answer is how opinionated you like your software. Well punk? How opinionated do you like it?

Programming, Python

Setting up a Mercurial work environment

So today I had a frustrating experience trying to get a Mercurial environment going for the current project I am working on. I am convinced for the kind of work we are doing the distributed branch model is going to be the right solution and I had been using a local Mercurial instance during my tech spiking earlier on the product.

Having built Python and Mercurial on the server and created a Mercurial user with the SSH keys of the individual developers I thought it would be easy sailing this morning. Not a bit of it and all of the problems stemmed from the lack of by default support for SSH in Windows.

Like most people I use the Putty Tool Suite for my SSH needs on windows. However simply aliasing putty to ssh doesn’t work. There are some key command switches that are different (including the trivial but annoying -P instead of -p for port).

Coming from a Java Subversion background I am used to having a portable library to take care of my ssh protocol needs. I also use OSX for my home development and that obviously provides a UNIX ssh command-line implementation, hence hassle-free SSH based source control.

The command line tool turned out to be the root of all my problems, both Eclipse and NetBeans Mercurial plugins don’t provide an implementation of the Mercurial client, instead they delegate it to the Mercurial client on the host OS, that in turns delegates ssh protocol invocations to the command line SSH.

The solution is simple once you know it, in the Mercurial.ini file you can alias ssh to Putty’s plink executable. There is in fact an example in the Mercurial book. Better still you don’t have to specify the key used if you are using Pageant.

However getting the first station to work was incredibly hard work. I even downloaded Cygwin just to get a more normal Unix ssh but by that point I had put a typo into the ssh alias and was getting a very weird error whenever I invoked Mercurial and I’d lost the forest in the trees.

At one stage I even gave Bazaar a go, hoping that SFTP might cure all my ills. However Bazaar uses Paramiko for it’s SSH support and on Windows that was failing due to its inability to find an OS source of entropy.

Server-side Python also let me down a bit as I was having an issue with the zlib module and after similar experiences with Ruby I knew that this would be because I must have compiled Python before zlib. Despite cleaning and re-configuring Python it still didn’t build the zlib module and in then end I had to go and run configure within the zlib module manually prior to a full Python build. This is exactly the same issue I have had with Ruby, what is the problem with this library?

Once you’ve done it once Mercurial repositories are actually easier to setup and manage than SVN (and that itself is pretty easy once you’ve done it a few times) and if you are working in a UNIX environment then they are extremely compelling.

However on Windows you are currently going to either do a lot of preparatory reading or be ready to set aside six hours to prevent frustration. In my case it probably didn’t help that I am also the one selling the utility of Mercurial. The pioneers always have the arrows in their backs; on the other hand they also get to better places long before anyone else.

Java, Python, Scripting

Sun recruits Jython leads

I know this isn’t exactly breaking news but it is further evidence that Sun is aiming to turn the JVM in a language agnostic platform. It’s also good news for the Jython project which has suffered a long period of hibernation and which has fallen far behind CPython in terms of its compatibility.

I like Python as a language and its clarity is great for one time scripts (which never really are). I would really like it to be a full member of the JVM-compatibles.

Groovy, Java, Python, Ruby, Scripting

Groovy or JRuby?

Martin Fowler blogged about the question a couple of days ago and ever since I have pondering that maybe it is not really the right question to ask.

I currently toodle between Jython, JRuby and Groovy for various reasons and I am an expert in none. The interesting thing I have found is that it is hard to pick one and just focus on that. To some extent they overlap heavily: they are all cross-platfrom, they are all dynamic, they all integrate with Java API stack I’ve committed to memory.

The first thing to say is that I am interested in scripting languages for prototyping and admin style scripting. I have never used Rails and the Grails data-model means that you need a specific kind of project to work on. If you want to use a particular product and that is only on one platform then that kind of makes your decision for you.

Each language has its own strengths, from my point of view I would categorise them in the following way. Jython has Python’s readability and solid language design, JRuby has Sun’s support, excellent community contributed library code and is very dynamic, Groovy is mini-Java, so it’s easy to learn and most importantly it has a functioning interactive console.

The last point might seem a bit weird, what about jirb and jython‘s interactive mode? Well Martin makes a very important point in his post about the purpose of these ports. Both JRuby and Jython aim to stay faithful to their source languages and be able to run code from their parent C implementations while expanding the API by accessing the Java libraries. Groovy on the other hand stays close to Java syntax and is the only one of the three that allows you to cut and paste code from a regular Java application and then play around with it in an interactive session. That is a very powerful and compelling feature.

Almost all the Groovy I do either comes from wanting to leverage or understand a piece of Java code.

All my Jython work on the other hand is about wanting to automate administration or manual tasks in a clear and concise fashion. Python’s dynamic data structures help, but so does zxJDBC the Jython specific database library that mixes DBI with JDBC to create a highly portable but simple database connectivity solution with no boilerplate!

JRuby on the other hand is something that only really comes up because Alpha Geeks love it. The syntax is gnarled and there is a significant learning curve before someone from a Java background can get “The Ruby Way” of things. The new integration of JRuby into NetBeans though makes developing in the language a comparative snap and I would suspect that JRuby will be a valid choice of application development language alongside Java now. The choice will be driven by the problems you are trying to solve not because one language is inherently “better” than the other.

Python, Scripting

Python Functional Programming

Something that is relatively new to me in Python is the lambda function that allows you to do a bit of functional programming. Today I created a Dictionary of functions and it all Just Worked. It reminded me a bit of C function pointers, Ruby closures and Java anonymous classes but it seemed more logical and natural than all of them.

Python, Ruby, Scripting

Starting Ruby and Python

This weekend I had a chance to start some Ruby and Python programming. My favourite scripting language to date is Perl and in terms of Java scripting I do actually like Groovy (although it has rapidly fallen out of favour with the Java crowd). Out of the two I have done currently done more complex stuff with Ruby but one thing is pretty immediate and that is that Python is very much on the back foot when it come to OOP scripting.

That’s not necessarily a bad thing though as actually scripting tends to be procedural and my Python script seemed to do a similar amount of work without the involvement of OO code. If I wanted something done quickly I would probably go for Python in the same way I go for Perl now. However if I thought something had legs and needed to be supported and expanded I would definitely go with Ruby.

One thing that both languages seem to have issues with is their big libraries. Most of my time with both languages was spent looking up how to perform some action. I was certain that neither of the languages required me to roll my own but I felt the situation was a lot like C and C++. C, like Perl, has the advantage of being syntactically very compact. You have to learn very little to know the language completely. Of course that means it lacks a certain power that comes from building on the capabilities of the language.

Perl answers this problem with modules which are a good idea as you only have to learn about standard libraries in little module sized chunks that match the problem you’re working on.

Python is kind of similar, in the sense there isn’t a lot in the core, but different in that, like C++ (and Java), you need to know your standard library if you are really going to get stuff done effectively.

Ruby is an altogether different beast. To get anything done you have to grapple with what feels, from my Java-centric point of view, like a very arbitrary API that is attached to every object. If you know the capabilities of a type then you have access to a lot of power. If you don’t, then you are swearing, scratching your head and reaching for the API description in the Pickaxe book.

One key thing that I feel is holding me back is the lack of variable typing. Having dynamically typed variables is cool, unless of course I know that I only want one type of variable (say an Integer). Maybe the code reflection and auto-completion features of Java are influencing me here but I do think it is helpful to be able to specify a type for a variable and then have the programmer’s editor or IDE be able to determine the API available to you.

Oh and yes, I might be making this complaint because I have already been caught out by the lack of automatic conversion between strings and numbers.

Overall my opinion at the moment is that Python is like Perl only cleaner and more legible. I think it would be really good for protoyping an application with a big team because there is a lot more consistency in the syntax and the style.

However Ruby’s object-orientation does mean that it would be an obvious choice for trying out OO Java ideas. Particularly if something like JRuby helped bring the benefits of both languages together (as well as making the most of my heavy investment in memorising huge chunks of the Java API).