Python, Web Applications

Deploying Python apps to Epio

I recently got my beta access to ep.io, the Python application deployment platform. I had the chance today to have a play around and try out some deployments so I thought I would try and give my view on the experience before. I’ve deployed Python apps to Heroku and Gondor before so those services form my reference points here.

So firstly, there’s a command-line client that you install via pip and you effectively deploy to the platform via a client-command, SSH keys and what looks like git on the server-side. This is more like Gondor than Heroku (which is intimately linked to git). It means you have your choice of source control and if you want to be a Python purist you never need to step outside of Python for everything you are doing.

Applications consist of essentially one configuration file that states where the WSGI application is and what the requirements file is. Compared to Gondor it is a very simple setup but it did feel that it could be even simpler if it made convention-based assumptions such as the requirements file being called requirements.txt, for example.

Leveraging WSGI and configuration this way gives a very flexible platform and I was able to get both Flask and Bottle to work (the former very quickly because it has documentation, the latter via trial and error that might require its own blog-post). I didn’t have time to try Django but I felt pretty confident that I could get whatever framework I wanted working once I understood the basic setup.

Unlike Heroku, Epio provides a fixed framework for executing the apps. It seems you will be running behind NGINX and Gunicorn. Both are good choices and I certainly like them but if you want to play around with different servers like Tornado or CherryPy you may prefer Heroku’s more open deployment model. I did like the way that you can use the configuration file to have NGINX serve static content directly.

Epio naturally has less of an ecosystem than Heroku but has Solr, Postgres and Redis out of the box. All solid choices and covering off the majority of what I would need. I was certainly grateful that I didn’t have to grapple with remote database administration and could prototype apps with just Redis.

Deployment and logging have kind of rough edges. Being able to access logs directly from the application page was a win for me, however when I was struggling to define the WSGI entrypoint correctly it seemed as if the application wasn’t being really compiled until the first request comes in. I would see an entry confirming a new deployment but then nothing until I hit the app. I think there should be some kind of sanity check of what you have uploaded to see whether it will even run.

Right now epio is providing a Python-based cloud deployment platform with a sensible set of supplementary services and low opinion about the source control system to you use. It feels like if this had been around at the start of the year it would have blown me away. However now there is more competition and therefore questions of price and ease of use will matter in terms of  how compelling it is to use the service.

If you do Python web development I would definitely recommend you sign up for beta and give it a go yourself as it seems a very solid prototyping platform. If you are not a Ruby and Git fan then you may well love what is on offer here because it is already very convenient, makes few demands on you and gets your web app public in minutes.

Standard
Work

Google Apps and App Engine

If you use Google Apps to provide you with email then you should also really be thinking about enabling and using Google App Engine as well. Internal applications are much easier to deliver to the business as a whole and having a ready-made platform makes it easier to try out ideas that previously would have been impractical.

The first advantage is that Google Apps that are bound into your domain allow you to create something that is easy to access for an existing user (no additional login is required) but also gives you peace of mind that you are exposing virtually zero surface area for attack.

The second is that for Python at least it is easy to access a very full featured environment with a minimum of code. Want to send emails, have task queues, access to memcache, serve static content? It is all a YAML configuration line or import away.

I love services like Heroku but a lot of internal apps have relatively light usage and benefit from the batteries included approach rather than combining various plugins. It makes it easy to switch between different approaches and react to different demands.

Standard
Work

Is this really the manifesto we want?

Silicon Milkroundabout tried to produce a manifesto for why people should consider working at a startup. This is the outcome.

The first time I saw it I was very disappointed. While I cannot knock its authenticity it is a profoundly depressing document. While there are the standard statements about passion and having the freedom to make what you should rather than what you are told to; there is much more about poverty, tiredness and scarcity.

If I read this I would say that working for startups is a mugs game. You’re far better coming in during the expansion phase when salaries are higher and the business case better proven.

The many references to tiredness and lack of sleep is also revealing. What I have discovered is that tremendous pressure is put on you to deliver product in a technology startup and this should be resisted at all costs. Sustainable pace is more important in small organisations than in large ones. In a large organisation you can actually burn out a team to achieve a goal because you probably have access to the resources to replace them. In a small one, once you’ve wrecked a team (probably including yourself) you have no way of replacing them and a death spiral will inevitably set in as decision making becomes progressively worse. Remember that a startup should aim to deliver progress not product. Don’t work with people who don’t understand this.

Money, frankly seems to be the missing ingredient from this list of reasons. Maybe adding “because late-stage equity options are worthless” would ruin the overall tone. Many people, especially investors, are involved in startups because they offer potential massive returns in a low growth environment. Americans are much more open and brash (you might even say vulgar) about this with talk of flipping and sale valuations of millions of dollars (often farcical as in the case of Groupon who merely had the bad luck to be caught before their IPO).

Even then this reason is foolish because if what you want is money then you should go to the City. The money is guaranteed, guaranteed in fact by the government which not only underwrites it, bails it out but then charges off into Europe to protect it from legislation that might affect its lucrative tax haven and money “recycling” business. In contrast being involved in “entrepreneurship” is a rather romantic and significantly more challenging way to achieve wealth.

I do work at a startup though and I was at Silicon Milkroundabout trying to encourage people to join me in doing this.

My personal motivation is that for me a startup is a business that is complete but small enough that you can actually see and understand all parts of it. The interesting thing is that organisational dysfunction is actually just a likely in a startup as a larger firm. Often the problems are actually exactly the same, simply orders of magnitude less significant.

Being able to pull the curtain aside is fascinating. Working in the small also removes the mystique that gathers around things and people that generate large revenues. Once a certain number of livelihood’s become involved in a particular process or product you lose the ability to tinker with things or even to question why things are the way they are. In an environment with no money and no customers any change is either positive or at least neutral.

Working in a startup for non-cynical reasons means creating something that is of profound personal interest. I really am interested in trying to remove friction from the process of turning ideas into reality. Wazoku is a product that I do believe in and what I was saying to a lot of people at Silicon Milkroundabout was try the product. If you are interested in solving the problem and the solution in turn solves some of your problems your work is satisfying at all levels. If there is not a satisfactory solution already in progress for your problem then a startup is the only way that you can initiate that process of moving to a more perfect world.

Working for a startup is a last resort; need should be part of your motivation; ignore idiots and their advice; make sure you get enough sleep.

Standard
Work

Silicon Milkroundabout Roundup

Interesting time at Silicon Milkroundabout this Sunday. There were kind of three levels of activity going on, first of all there was the element of developer goofing off with arcade machines and free stuff. Then there was the opportunity to network, first of all between the startups and secondly between the developers (although I am not sure how much mixing between different dev teams was actually going on).

Finally there was the recruitment activity. Unlike the first event this really was more of a milkround with a younger, less experienced audience. The format did seem to be pitching for talent which is interesting as I am not convinced that people are going to find the best role by going with the best sales pitch. There has to be a better way of understanding the culture of the firm you are potentially joining.

The different streams of activity make the event quite weird in its nature and purposes. It feels like there is a need for a kind of startup expo to allow startups to see and meet one another without the pretext of seeking to employ people. There is also a need for a kind of elite coder event on a quarterly basis that is maybe a little select, a bit like a mini-conference, that allows for networking and swapping of intelligence and gossip on what is really going on at various firms.

Standard
Work

The Pretentious CTO

I currently use the title CTO in my job despite the fact that I only directly manage two people. A classic example of the “fake” CTO. So naturally I felt a little defensive in a recent discussion with Jon Hartley about why I feel that the title can be justified by people who work in small businesses.

Let’s start with an entirely pragmatic answer: the job title is something that is well understood. While the majority of my activities day to day would be adequately covered by the title “lead developer”, the truth is that technical authority and decision making resides solely with myself. The easiest way to convey that to suppliers and recruitment agents (and stop them seeking to go over my head to my non-existent boss) is to use the most commonly understood title.

I find it ironic that I have managed much larger projects with a much junior title. In terms of experience I do not feel a particular gap with other startup CTOs but obviously there is a bigger gap as you move into the equivalent role in larger organisations. A lot of those people come from non-technical backgrounds reflecting the greater need for people management at larger scales. The number of people who have technical backgrounds and have managed groups greater than a hundred people strong onshore are probably pretty small.

For me the key differentiator in my current job is that I do hold a technology portfolio within the management and report on the whole technical area to the management team as well as the board and investors (although to be honest the latter too are not that bothered so far). I would happily concede the “Chief” as I have no other senior technology reportees but I think it is a different kind of pretension that seeks to do down the unique aspects of a role you have in an organisation. I am an officer of the company and I do make the key decisions for technology and I am held to account for them. CTO is the common title and I am comfortable using it.

Standard
Programming

Google Chrome wants to sit at the table

I fixed a weird CSS bug last week which I think is an interesting illustration of the difficulties of implementing a standard. One of the ways that I layout our website is by using CSS Tables to provide right-sidebars and content splits. One of our components consists of an image in a left-column, content and then a button bar in a right-column.

The whole strip of content was marked as display: table-row and the columns were display: table-cell.

On Firefox this was fine and the full width of the page was used with the correct proportions for the content blocks. On Chrome however the width of the whole component was around 75% of the whole page width (with the correct proportions). After a lot of examination it appeared that Chrome genuinely felt the page was narrower at that point.

It took a few weeks to get the necessary mindshift to fix it. The problem was the interpretation of how display: table-* works when you don’t have the full hierarchy. My interpretation is that the browser fills in the implied parent elements and that worked correctly in Firefox. In Chrome however I needed to change my table-row to table for the full width of the page to be used. The implied table wasn’t the full width of the page, while the explicit one was with the implied table-row occupying the correct full width.

This worked fine on Firefox as well so the problem was solved! Always have a top-element that defines display: table.

Standard
Programming

I need feedback not features

“How about LDAP? Or a configurable workflow?” opines the new salesman. “I think it would really help the product if it had help videos” chirrups the new intern.

All interesting ideas I’m sure and some of them might be relevant in the future, but to be honest I didn’t really need a lover here I just needed a friend.

A lot of people seem to confuse feedback with the opportunity to point out the depressingly long list of things that an application doesn’t have, doesn’t have completely or just doesn’t do. As a developer it often feels that there is a never-ending queue of people ready to point out that the kettle washes clothes really badly.

Offering new features is actually a substitute for genuine feedback, which for me is discussing what the application does do on its own merits. In particular one thing that people never seem to be good at is to point out how existing features can be leveraged in multiple contexts or in different ways to maximise their benefit. Charles Vasey, a boardgame designer is amazing at doing this kind of refinement and a big inspiration here. Instead I seem to be left to sweat the codebase on my own while people boot more and more stuff into the backlog (blithely ignoring their own self-defeating behaviour).

A natural instinct (and one I’ve been succumbing to recently) is just to ignore everything everyone says unless it is a bug report. One particularly strong point in favour of doing this is that since I have a backlist that goes into October why not wait until that is clear and then gather feedback? I too already have a massive list of features I wish I could add but have discounted on their lack of value, surely I would be better off implementing them first instead of soliciting feedback on something that is incomplete.

However realistically, retreating into the Ivory Tower is rarely the best option. But how can we move from features to feedback? I think perhaps the first step is to call out the behaviour and just park each new feature onto a Post-It and then re-iterate the need for feedback. The other idea is make people explicitly say what they like and then why they like it. Forcing to people to focus on the positive and make them express their response in terms of what they want to see more of rather than what they would like to see.

Standard
Java

Delta versus Chain

There are already two styles of persistence code in my daily work codebase. The first is based on the concept of generating the delta of the change you want to make. The second is the construction of the new data map by chaining functions together.

Let’s take some examples:

delta = { 'when' : now(), 'comment' : 'Hulk smash!', 'tags' : tags.append(new_tags) }
comment.update(delta)
store.save(comment)

store.save(update_timestamp(comment(update_tags(new_tags), 'Hulk smash!)))

Hopefully you can already see some of the pros and cons about both approaches. The chaining (or trainwreck style) ends up being very short and powerful. However when you need to pass in a few parameters at different points in the chain it can get very complicated to read.

The delta pattern allows you to accumulate changes into the delta neatly and to combine several changes (data, timestamps, audit entries) into one atomic write that is, in my view, easier to read than the chained example. However the boilerplate doesn’t go away and starts to get irritating after a while.

These are Python examples because in Clojure the threading macros and argument lists take away my comprehension concerns.

At the moment I’m continuing to experiment with both strategies and looking for a way to incorporate the best elements of both the comprehension of the delta and the conciseness of chaining.

Standard
Clojure

Why I’m finding Clojurescript underwhelming

I noticed Clojurescript in Github before the big announcement and thought it was an interesting idea. I am a big fan in general about having a Clojure syntax that compiles to Javascript. As a platform it is even more ubiquitous than Java and it would be a great way of simplifying Javascript’s closure and function syntax.

However in practice Clojurescript has been desperately disappointing for me. Firstly there is the weird decision to not have the code run on OpenJDK. This really limits its utility: I don’t seem to have a machine with a compatible setup at the moment despite having various flavours of Javascript interpreters available.

Then while looking for an answer as to how soon this problem is likely to be resolved I discovered this thread which was another level of disappointment. The original post is undiplomatic, perhaps even inflammatory, however the response indicates a level of befuddling clueless-ness.

If you want something to compile into Javascript I think you actually do want it to compile into good idiomatic Javascript unless you have a really good reason not to. You also do want to be able to use really good existing frameworks like jQuery (which really is the defacto standard right now).

The reason I think these are reasonable requests is that Coffeescript seems to manage to do both. Before Coffeescript maybe Clojurescript’s idiosyncrasies would have been forgiveable but being late to the party as well as being less well-mannered makes the defiance in the response seem poorly judged.

I am not sure what Clojurescript is really for (apparently it is aimed at a future community of people that don’t exist yet, which is … helpful). I don’t feel that it is really simpatico with the existing Javascript code that works in the browser and I am not sure it really has a place in the server-side world of Node.js where it might have been a better fit.

I remain open-minded though and would be willing to give Clojurescript a second go once the dust has settled a bit.

Update: I’ve written a follow up to this post

Standard
Web Applications, Work

Using SVG in the modern website

Using SVG when you are putting together a new website is a pretty sound decision, it’s over a decade old, well-supported by browsers and the ability to scale images accurately via CSS is pretty compelling when you are rapidly trying out different layouts and proportions.

Of course until recently IE has been the bugbear but IE9 actually has pretty decent SVG support. It is now worth thinking of using SVG as the general case and IE8 as the exception which can be switched to PNG via Javascript. The first iteration of Wazoku Idea Spotlight used SVG exclusively and the second iteration will do a Modernizer based switchout for IE8 but essentially still be SVG based.

Therefore I was pretty confused when I was taking a random check at the app in IE9. Instead of displaying alt text or the images instead there was just whitespace. Quickly opening the images revealed that IE was quite happy to render them at full window size and that there was no issue with loading them.

After some confused Googling I found out that the issue was that the previous generation of SVGs were generated straight out of Adobe Illustrator where as this set are going through Inkscape where I am tweaking the colour, size and so on. Inkscape does not allow you by default to specify a property called the viewbox. Instead this is only created if you export your file as an Optimized or Plain SVG. It is an outstanding feature when you go looking through the Inkscape bug list but it is a really obscure bug (hence this blog) to track down. The reason the images were appearing as blank is that without a viewbox IE9 crops the image to the CSS dimensions rather than scaling it. Firefox and Chrome scale it as you would expect. Essentially I was seeing the top-left 32 pixels of an image that IE9 considered to be 640px square, overflow hidden.

Having found the problem I then converted a test image to Optimized SVG, who doesn’t love Optimized things after all? Well the answer to that is Chrome. Firefox (probably due to having the longest SVG heritage) did the right thing in both cases and IE9 was fine with the Optimized version. Chrome stretched the image out on the vertical and via the Developer tools it was possible to see that the Dimensions value for the image was completely incorrect with a letterbox set of dimensions rather than a square.

In the end the thing that worked everywhere was Inkscape’s Plain SVG format. Something I am fine to live with. It would be nice to be able to set a viewbox from Inkscape’s Document Properties though and I will be keeping an eye out for it on the release notes in future.

Standard