Programming

Slow SPAs are worse than NoSPA

I got a digital subscription to the Economist for my birthday last month so I’ve started reading a lot more content on their site. As a result I’ve noticed a lot of weirdness with their page loads that was hardly noticeable when I was using the free tier of a few articles per week.

The site seems to be built as a SPA with a page shell that loads quite quickly but takes far longer to fill with content and which has some odd layout choices and occasional pops and content shifts.

The basic navigation between the current issue index and the articles is hampered by what appears to be a slow load or render phase. Essentially it is hard to know whether the click on a link or the back button has registered.

By replacing traditional page navigation the experience is actually worse. The site would be better if the effort going into the frontend went into faster page serving.

I’m not sure if the page is meant to be doing something clever with local storage for offline use but it seems to need to be connected when browsing so I’m assuming that this is something to do with the need for a subscription and payment gateway that prevents a fast server load of content.

It still feels as if the page and the 200 words or so should be public and CDN-cached with the remaining content of the article being loaded after page-load for subscribers.

The current solution feels like someone has put a lot of effort and thought into making someone that is actually worse than a conventional webpage and that seems a shame for a site with relatively little content that is mostly updated once a week.

Standard
Gadgets

The HP Stream as a cheap open-source Chromebook

I really like Chromebooks but I must admit there are times when I’m travelling and I’d like a more conventional offline setup.

Also one of the attractions of the original wave of Chromebooks were their price and I feel that the more recent editions have been price comparable to mid-range laptops.

Finally I read a few pieces warning against slipping into a monopoly of convenience around the Chrome browser so I thought it would be great to have a Chromebook that essentially ran Firefox.

I chose the HP Stream as the cheapest laptop that people mentioned that had Linux running on. It’s plastic and it’s proud of it, I bought one in an aquamarine blue. Trying one out in a store it has a good sized keyboard with only one quirk around the enter key which a bit smaller than the typical UK keyboard and has a hash key right above it.

That plastic shell means it is light but it is durable to have been shoved in a few travel bags and transported round. It has a custom power adaptor which is a pain when you’re travelling. USB-based charging would have been much better.

Installing Linux

I chose Elementary as my distribution, partly out of curiosity but also because people mentioned that it was an opinionated take on what a user-friendly operating system might be.

I had to read how to enter the Stream’s BIOS on the internet and set it to boot from a USB but after that it was pretty plain sailing to get the Elementary ISO onto the USB and for the Stream to load it.

Everything worked first time and I had the new OS completely replace pre-installed Windows and also encrypt the drive, which seems important if you are going to be travelling with the laptop and thus more likely to lose it or have it stolen.

Connecting to Wifi was painless as was the upgrade process.

Elementary

Elementary takes a very different approach to all the other Linux distributions with the UI essentially hiding all the normal Linux details. It has its own app store and upgrades of managed packages are seamless and mostly seem to complete without the need for restarts.

You can get to the terminal and install packages via Debian-style packages if you want to but Elementary mostly shines when you stick to the GUI and the apps.

For the kind of consumer electronic experience I wanted Elementary was an excellent choice. The only real problem I encountered was the classic issue of the OS not suspending when the laptop lid was closed, instead I had to manual suspend and then close the lid. This has subsequently been fixed in a subsequent update and currently the whole OS feels like what you’d expect from a modern system.

Performance

I am an absolute tab monster when it comes to browsers. I rarely have less than 40 tabs open in each browser and I have been known to use multiple windows on top of this.

My Chromebooks have struggled in the past and I’ve had to use the Task Manager to figure out where the CPU and memory are going in terms of the sites I use.

The situation was similar with the Stream. When I run Chrome and Firefox the system struggled and the battery ran flat quickly. I’ve since stuck to just Firefox and tried to limit myself to a set of about 15 to 20 tabs. This has been stable and has given me a few days worth of web browsing per charge.

Conclusions

When I started experimenting it would have been hard to say that I really had a better experience using my setup compared to a Chromebook. As we head towards Christmas, almost a year into the experiment, I’m now starting to think that I’d prefer to take the machine that would allow me to use git and my choice of text editor if I wanted. Firefox is a great choice for most of my daily websites and both of the machines have encrypted drives.

Elementary isn’t quite consumer electronics and the Stream is probably the wrong side of the price/quality divide but its actually the flexible machine I want to talk as I visit family and friends over the holidays.

Standard
Work

Please stop showing me “the data”

This isn’t a reactionary rant against data-driven decision making and it isn’t about nostalgia for gut-driven benevolent dictators.

Instead it is an appeal for reason to play an equal part in decision making.

The seed of this post was planted by a keynote Ines Montani gave at EuroPython. At the time I was more interested in her central argument that paying customers are the most important metric a business can have.

But in part of the talk she talks about the cliche of “show me the data”, a phrase that I think originated at NASA where, in context, it makes a lot of sense but when transplanted to the world of small business quickly becomes expensive, slow and farcical.

In part of her talk Ines mentioned that when making decisions on how to run a small business there shouldn’t be a need to provide data for or against every decision. “Why can’t we use reason?” she asked.

The question had huge resonance for me. The emphasis on data-driven decisions in businesses has not led to improved data or statistical literacy. Instead it has led to the generation of fig-leaf numbers, impenetrable spreadsheets of data as obfuscation and irrelevant but voluminous data collection. I see little evidence that decision-making is better.

It has also exposed the idea that the problem is data collection. The more information we collect then the more it feels like the more any decision can be justified or any course of action advocated or vetoed. Interpretation, selection and analysis of the data is more important than ever, and this at its heart requires reasoning.

Reason is different from “common sense” in that it should be produce self-consistent decision making that can be justified and interrogated. Reasoning is a process applied to instinct, insight, intuition, experience and knowledge.

So please don’t show me your data, explain your decision instead.

 

Standard
Programming

PyPika

PyPika is a DSL for creating SQL queries. It works by generating SQL from definitions that you create inline rather than by interpreting models or data classes or structures.

It makes it easy to define minimum projections and is also straight-forward to bind data to as you simply provide the values into the query that is generated rather than binding values to a query definition.

Once you have the query object constructed you call str on the object to obtain the actual SQL which you then need to pass to some execution context.

This also means that it is easy to test what SQL you are generating (although really using a DSL like this means you should really be trusting the library).

I have only had one real problem with the statements I have used to date and that is around the Postgres `RETURNING` statement that allows a generated key to be returned to the caller of an INSERT.

Apart from this using Pypika has been better than using a Python template or writing raw SQL.

Standard
Programming

Sustainable web development

This article about delivering a sustainable small website is one of the most inspiring things I’ve read in weeks. It made me think about the true costs of the web revolution and made me think again about the environmental claims we make about moving paper processes online. Green energy and clean websites are entirely possible but will people pay the premium for it?

Finally like all truly great pieces it offered some practical insight into problems I have been grappling with in my work. The observation that small, fast loading server-rendered pages are the same as client-side SPAs but more energy efficient is a great insight.

However the best was the idea of simplifying the navigation into its own page. This one thing is a piece of genius that hundreds of man-years of UX research have failed to discover.

Thinking about the energy cost of our websites is also a great proxy for thinking about the cost basis for our infrastructure and where we want to put our processing effort.

Standard
Programming, Software, Web Applications, Work

Prettier in anger

I’ve generally found linting to be a pretty horrible experience and Javascript/ES haven’t been any exception to the rule. One thing that I do agree with the Prettier project is that historically linters have tried to perform two tasks to mixed success: formatting code to conventions and performing static analysis.

Really only the latter is useful and the former is mostly wasted cycles except for dealing with language beginners and eccentrics.

Recently at work we adopted Prettier to avoid having to deal with things like line-lengths and space-based tab sizes. Running Prettier over the codebase left us with terrible-looking cramped two-space tabbed code but at least it was consistent.

However having started to live with Prettier I’ve been getting less satisfied with the way it works and Prettier ignore statements have been creeping into my code.

The biggest problem I have is that Prettier has managed its own specific type of scope creep out of the formatting space. It rewrites way too much code based on line-size limits and weird things like precedent rules in boolean statements. So for example if you have a list with only one entry and you want to place the single entry on a separate line to make it clear where you intend developers to extend the list Prettier will put the whole thing on a single line if it fits.

If you bracket a logical expression to help humans parse the meaning of the statements but the precedent rules mean that brackets are superfluous then Prettier removes them.

High-level code is primarily written for humans, I understand that the code is then transformed to make it run efficiently and all kinds of layers of indirection are stripped out at that point. Prettier isn’t a compiler though, it’s a formatter with ideas beyond its station.

Prettier has also benefited from the Facebook/React hype cycle so we, like others I suspect, are using it before it’s really ready. It hides behind the brand of being “opinionated” to avoid giving control over some of its behaviour to the user.

This makes using Prettier a kind of take it or leave it proposition. I’m personally in a leave it place but I don’t feel strongly enough to make an argument to remove from the work codebase. For me currently tell Prettier to ignore code, while an inaccurate expression of what I want it to do, is fine for now while another generation of Javascript tooling is produced.

Standard
Programming, Python

403 Forbidden errors with Flask and Zappa

One thing that tripped me up when creating applications with Zappa was an error I encountered with form posting that seems to have also caught out several other developers.

The tldr is that if you are getting a 403 Forbidden error but your application is working locally then you probably have a URL error due to the stage segment that Zappa adds to the URL of the deployed application. You need to make sure you are using url_for and not trying to write an absolute path.

The stage segment

Zappa’s url structure is surprisingly complicated because it allows you to have different versions of the code deployed under different aliases such as dev, staging and production.

When running locally your code doesn’t have the stage prefix so it is natural to use a bare path, something like flask.redirect(‘/’) for example.

If you’re using the standard form sequence of GET – POST – Redirect then everything works fine locally and remotely until the raw redirect occurs remotely and instead of getting a 404 error (which might tip you off to the real problem more quickly) you get a 403 forbidden because you are outside the deployed URL space.

If you bind a DNS name to a particular stage (e.g. app-dev.myapp.com) then the bare path will work again because the stage is hidden behind the CloudFront origin binding.

Always use url_for

The only safe way to handle URLs is for you to delegate all the path management and prefixing to Zappa. Fortunately Flask’s in-built url_for function, in conjunction with the Zappa wrapper can take care of all the grunt work for you. As long as all your urls (both in the template and the handlers) use url_for then the resulting URLs will work locally, on the API Gateway stages and if you bind a DNS name to the stage.

If this is already your development habit then great, this post is irrelevant to you but as I’ve mostly been using Heroku and App Engine for my hobby projects I’d found myself to be in the habit of writing the URLs as strings, as you do when you write the route bindings.

Then when the error occurred I was checking the URL against my code, seeing that they matched and then getting confused about the error because mentally I’d glossed over the stage.

Standard