Category Archives: Programming

London, Programming, Web Applications, Work

Halfstack on the Shore(ditch) 2022

This is the first time the conference has been back at Cafe 1001 since the start of the Pandemic and my first HalfStack since 2021’s on the Shore event.

In some ways Halfstack can seem like a bit of an outlandish conference but generally things that are highly experimental or flaky here turn up in refined mainstream forms three to five years later. Part of the point of the event is to question what is possible with the technologies we have and what might be possible with changes that are due in the future. Novelty, niche or pushing the envelope talks are about expanding the conversation about what is possible.

The first standout talk this year was by Stephanie Shaw about Design Systems. It tries to make the absurdist argument that visual memes meet all the criteria to be a design system before looking at what are the properties of a good design system that would disqualify memes. The first major point that resonated with me was that design systems are hot and lots of people say they have them when what they actually have are design principles, a component library or an illustration of UI variant behaviour.

I was also impressed that the talk had a slide dedicated to when a design system would be inappropriate. Context always matters in terms of implementing ideas in organisations and it is important to understand what the organisation needs and capabilities that are required to get value from an idea. Good design systems provide a strong foundation for rapid, consistent development and should demonstrate a clear return on the investment in them.

One of the talks that has stayed with me the longest was one that was about things that can be done now. I’ve seen Chris Heilmann talk about dev tools at previous conferences but this time the frame of the talk was different and was about using dev tools in the browser to make the web sane again. He reminded me that you can use the dev tools to edit the page. Annoying pop-up? Delete it! Right-click hijacked? Go into the handler bindings and unbind the customer listener. Auto-playing video? Change it’s attributes or again just delete the whole thing. He also did explain some new things that I wasn’t aware of such as the ability to take a screenshot of a specific node from within the DOM inspector. I’ve actually used that a few times since in my work.

There was an impromptu talk that was grounded in a context that was a little hard to follow (maintaining peer to peer memes in a centralised internet apocalypse I think) but was about encoding images into QR codes that included an explanation of how QR codes actually work and encode information (something I didn’t know). The speaker took the image data, transformed it into a series of QR codes, then had a website that displayed the QR codes in sequence and a web app that used a phone camera to scan the codes and reassemble the image locally. The scanning app was also able to understand where in the sequence the QR code was which created a kind of scanning line effect as it built up the image which was very cool to watch.

There were three talks that all involved a significant amount of simultaneous interaction and each using slightly different methods but clearly the theme was having many people together on a webpage interacting in near real time.

The first thing to say is that I took a decent but relatively low-powered Pinebook laptop to the conference as I thought I would just need something simple to take notes and look things up on the internet, maybe code along with some Javascript. All of the interactive demos barely worked on it and the time to be active was significantly longer than say the attendees with the latest Macs. I think the issue was a combination of having really substantial downloads (which appeared not to be cached so refreshing the browser was fatal) but also just massive requirements on CPU in the local synchronisation code.

The first was by a pro developer relations person, Jo Franchetti, who works for Ably and who used the Ably API. Predictably this was the best working (and looking) demo with a fun Halloween theme around the idea of an ouija board or, more technically, trying to spell out messages by averaging all the subscribers’ mouse movements to create a single movement over the screen. However even using a commercial API, probably having no more than 25 connections and a single-screen UI my laptop still ground to a halt and had significant lag on the animations. It did look great projected on the big screen though.

Jo’s talk introduced me to an API I hadn’t heard of before scrollTo (part of a family of scrolling APIs). This is an example of how talks about things on the edge of the possible often come back to things that are more practical day to day.

James Allardice and Ross Greenhalf had the least successful take on the multiuser extension and in terms of presentation style seemed to be continuing an offstage squabble in front of everyone. I get the impression that they were very down on what they had been able to achieve and were perhaps hoping for a showcase example to promote their business.

Primarily they didn’t get this because they were bizarrely committed to AWS Lambda as the deployment platform. Their idea was to do a multiplayer version of Pong and it kind of worked, except the performance was terrible (for everyone this time, not just me). This in turn actually created a more fun experience that what they had intended to build as the lag meant you needed to be quite judicious in when you sent your command (up or down) to the server as there was a tendency to overshoot with too many people sending commands as ball approached and then another as they were waiting for the first one to take effect. You needed to slow down your reaction cycle and try and anticipate what other people would be doing.

The game also only lasted for the duration of a Lambda timeout of a single execution run as the whole thing was run in the execution memory of a single Lambda instance. This was a consequence of the flawed design but again it wasn’t hard to imagine how Lambda could be quite effective here as long as you’re not using web sockets for the push channel. It feels like this kind of thing would probably be pretty trivial in something like Elixir in a managed container but was a bit of a uphill battle in a Javascript monolith Function as a Service.

The most creative multi-user demo was by Mynah Marie (aka Earth to Abigail who has been a performer at previous Halfstacks) who used Estuary to create a 15 person online jam session which was surprisingly harmonious for a large group with little in the way of being able to monitor your own sound (I immediately had more empathy for any musician who has asked the desk for less drums in their monitor). However synchronisation was again a big problem, not only did other people paste over my loops but also after leaving the session one of my loops remained stubbornly playing until killed by the admin despite me not being able to access the session again, I was given a new user identity and no-one seemed able to reconnect with the orphan session.

Probably the most mindblowing technical talk was by Ulysses Popple about his tool Nodessey which is both a graph editor or notebook and a way to feed values into nodes that can then visualise the input they are receiving from their parent nodes. It reminded me a bit of PureData. I found following the talk, which was a mixture of notes and live-coded examples, a bit tricky as its an unusual design and trying to follow how the data structure was working while also trying to follow the implementation was tricky for me.

One thing I found personally interesting is that Nodessey is built on top of a minimal framework called Hyperapp which I love but have never seen anyone else use. I now see that I have very much underestimated the power of the framework and I want to start trying to use it more again.

Michele Riva did a talk about the use of English in programming languages which had a helpful introduction to programming languages that had been created in non-English languages. As an English speaker you tend to not need to ever leave the US-led universe of English based languages but it was interesting to see how other language communities had approached making programming accessible for non-English speakers. There was a light touch on non-alphabetic languages and symbolic languages like J (and of course brainfuck).

Perhaps the most practical talk of the conference was by Ante Barić around browser extensions. I’ve found these really valuable for creating internal organisation tooling in a very lightweight way but as Chris Heilmann reminded us in his talk too many extensions end up hammering browser performance as they all attempt to intercept the network requests and render cycle. The talk used a version of Clippy to create annoying commentary on the websites you were visiting but it had some useful insight into what is happening with browser extensions and future plans from both the Google and Mozilla teams as well as practical ways to build and use them.

Ante mentioned a tool that I was previously unaware of called web-ext that is a Mozilla project but which might be able to build out Chrome extensions in the future and gives a simplified framework for putting together extensions.

General notes

Food and drink is available when you want it just by showing the staff your conference lanyard. Personally I think it is great when conferences are able to be so flexible around letting people eat when they want to and avoiding the massive queues for food that typically happen when you try and cram an entire conference into a buffet in 90 minutes. I think it also helps include people who may have particular eating patterns that might not easily fit into scheduled tea and lunch breaks. It also makes it feel less like school.

In terms of COVID risk, the conference was mostly unmasked and since part of the appeal is the food and drink I felt like I wasn’t going to be changing my risk very much by wearing a mask during the talk sections. The ventilation seemed good (the room could be a bit cold if you were sitting in the wrong place) and there was plenty of room so I never had to sit right next to someone. This is probably going to remain a conference that focuses on in-person socialising and therefore isn’t going to appeal to everyone. Having a mask mandate in the current environment would take courage. The open air “beach” version of the conference on the banks of the Thames would probably be more suitable for someone looking to avoid indoor spaces.

Going back?

Halfstack is a lot of fun and I’ve booked my super early-bird for this year I think it offers a different balance of material compared to most web and Javascript conferences. This year I learnt practical things I could bring to my day job and was impressed by what other people have been able to achieve in theirs.

Standard
Programming, Software, Work

Defining the idea of “software engineering”

I have been reading Dave Farley’s Modern Software Engineering. Overall it’s a great read and thoroughly recommended (I’m still reading through it but I’ve read enough to know it is really interesting and a well-considered approach to common problems in development).

One of the challenges Dave tackles is to try and provide a definition of what software engineering actually is. This is actually a pretty profound challenge in my view. I’ve often felt that developers have usurped the title of engineer to provide a patina of respectability to their hacky habits. Even in Dave’s telling of the origin of the term it was used to try and provide parity of esteem with hardware engineers (by no lesser figure than Margaret Hamilton).

In large organisations where they have actual engineers it is often important to avoid confusion between what Dave categorises as Design and Production engineering. Software engineering sits in the world of design engineering. Software is malleable and easy to change unlike a supply chain or a partially completed bridge. Where the end result of the engineering process is an expensive material object Dave points that it is common to spend a lot of time modelling the end result and refining the delivery process for the material output as a result of the predictions of the model. For software to some extent our model is the product and we can often iterate and refine it at very low cost.

Dave proposes the following definition of engineering:

Engineering is the application of an empirical, scientific approach to finding efficient, economical solutions to practical problems.

Dave Farley, Modern Software Engineering

This definition is one I can live with and marries my experience of creating software to the wider principles of engineering. It also bridges across the two realms of engineering, leaving the differences to practices rather than principles.

It is grounded in practicality rather than aloof theories and it emphasises that capacities drive effective solutions as much as needs. This definition is a huge step forward in being able to build consensus around the purpose of a software engineer.

Standard
Programming

State of the Browser 2022

I’ve attended a few of these conferences and have always found them helpful. This year it had relocated to the Barbican Centre with the food and drink area overlooking the beautiful Conservatory there, great choice as a venue.

The conference was a hybrid in-person/online event that I think could serve as a model for other conferences that seem to only be focusing on their return to in-person. Due to other commitments I wasn’t able to be at the venue all day and so at lunchtime I headed home and picked up a few of the rest of the talks on the livestream. It was great to have the flexibility and made the whole conference more accessible.

Talks-wise it was interesting as ever and a little bit less inward looking or niche interest that it has been in the past. There were the usual mix of upcoming standards and challenges in implementing them, how to apply techniques to the current broad mainstream of browsers and a little bit of evangelism for playfulness and environment impact.

One of my key takeaways was on this last point; using an image CDN that can do automatic content negotiation to use an efficient modern image standard has a huge carbon saving. It feels a bit crazy that so many companies are still serving fixed sizes and formats off things like Cloudfront and S3.

Bruce Lawson kicked off the event with a good historical perspective talk on the history of standards (and the struggle to create and maintain them) and brought the issues of standardisation through the search for technical solutions to the world of regulation and better digital policy. Engaging with law makers is a more realistic way to improve the online world that the search of technical solutions to social problems.

More practically we can hope that Apple will be compelled as a digital gatekeeper to allow competition on browser implementations on its platform and maybe even fund its Safari team properly to have better compatibility with the general web standards on iOS. I felt it was nice for a recognition that government organisations can be engaged and willing to listen and that progress can be made be working together rather than outside of regular power structures.

Probably the best talk I heard was “Be the browser’s mentor not it’s micromanager” by Andy Bell this talk neatly encompassed two major ideas: the first was the way that layout systems in CSS have advanced to the point where you are describing structure and allowing the layout manager to actually decide the rendering and secondly on how digital design approaches have managed to fall between the abstractions of the grid system and the precise layout of magazine style layout.

By leaning on the layout engines the amount of CSS we have to write is much more minimal than the micromanaging fussiness typical to component design systems. It is also more powerful and expressive, avoiding the overly complex muddle that is often associated with component style systems but also not going too far down the class frenzy of utility class systems.

Sophie Koonin taught me how to use the prefers-reduced-motion preference via the medium of late 90s website chaos. A good example of the mixture of fun and practical content.

I also enjoyed Alistair Shepherd‘s talk which had a few technical bits and pieces but managed to bridge the themes of the conference by wanting to create a personal website that first and foremost reflected his personal interests and then used the tech to deliver the vision he had for himself. Although the idea to have websites that vary according to the time of date is quite an interesting idea.

I didn’t catch the last few talks so I’m hoping to be able to watch them when they come to YouTube (or maybe some federated alternative!).

Overall still one of the necessary conferences to catch for web technology and now easier to engage with than ever before.

Links

Standard
Programming

Keeping the batteries included

Python is well-known for being a “batteries included” language which means it comes with a rich variety of modules that work right out of the box. However this recent post about the regular expression library shows the problems that can occur from shipping such a wide variety of code as part of the language core. Maintaining a wide-ranging codebase is challenging as is keeping it aligned to a language’s release cadence.

The problems suggest that really the language should focus just on the core tools and syntax of the language with everything else being on a different release cycle. However by itself that core isn’t too useful for pragmatic coding purposes. Curation of code and having a sensible selection of libraries is a challenge. Some languages like Clojure and Elm have a controlled ecosystem of libraries that are adjacent to a small language core. However here the difference between curation and gatekeeping is fine and it feels like only languages with a large community can do it effectively.

Perhaps the answer is to have a basic implementation of core functionality in the core but to use the language documentation to suggest alternative libraries. This moves the problem to the documentation team but hopefully this is a simpler arena to both maintain and curate.

Standard
Programming

PR Reviews: not the messiah, not a naughty boy

On Tech Twitter there seems to be a lot of chat recently about how the Github Pull Request (PR) review process is “broken” or underserves internal teams (an example) that have strong collaboration practices such as pairing or mob coding.

Another objection raised is the idea of internal gatekeeping within development groups, I’m not sure I fully follow the argument but I think it runs along the lines that the PR review process allows powerful, influential members of the group to enforce their views over the others.

This is definitely a problem but frankly writing AST tools linked to the “merge to master” checks is probably a more controlling tool than spending your time policing PRs.

With open source projects often all the action happens at the pull request because the contributors often have no other interaction. Proponents are right to point out that if this works for open source projects do you really need to put effort into other practices upstream of the PR? Opponents are also right to point out that adopting the work practice of an entirely different context into your salaried, work context is crazy. They are both right, individual organisations need to make deliberate decisions using the challenges of both side to the way they are working.

I’m not sure how much of this is a fightback by pairing advocates (that’s how I interpret this thread). There is a general feeling that pairing as a practice has declined.

In my work the practice is optional. As a manager I’ve always known that in terms of output delivery (which before people object, may be important particularly if you’re dealing with runway to meet salary) pairing is best when you’re increasing the average rather than facilitating the experienced.

I think even with pairing you’d want to do a code review step. Pairs are maybe better at avoid getting into weird approaches to solutions than an individual but they aren’t magical and if you are worried about dominant views and personalities pairing definitely doesn’t help solve that.

So I’d like to stick up for a few virtues of the Pull Request Review without making the argument that Pull Requests are all you need in your delivery process.

As an administrator of policy who often gets asked about Software Development LifeCycles (SDLC) as a required part of good software governance. It is handy to have a well documented, automated review process before code goes into production. It ticks a box at minimum disruption and there isn’t an alternative world where you’re just streaming changes into production on the basis of automated testing and production observability anyway.

As a maintainer of a codebase I rely on PRs a lot as documentation. Typically in support you handle issues on much more software than you have personally created. Pairing or mob programming isn’t going to work in terms of allowing me to support a wide ranging codebase. Instead it’s going to create demand for Level 3 support.

Well-structured PRs often allow me to understand how errors or behaviour in production relate to changes in code and record the intent of the people making the change. It makes it easier to see situations that were unanticipated or people’s conception of the software in use and how that varies from actual use.

PR review is also a chance for people outside the core developers to see what is happening and learn and contribute outside of their day to day work. Many eyes is not perfect but it is a real thing and people who praise teaching or mentoring as a way to improve technique and knowledge should be able to see that answering review questions (in a suitable form of genuine inquiry) is part of the same idea.

PRs also form a handy trigger for automated review and processes. Simple things like spelling checks allow a wider range of people to contribute to codebases fearlessly. Sure you don’t need a PR to use these tools but in my experience they seem more effective with the use of a stopping point for consideration and review of the work done to date.

Like a lot of things in the fashion-orientated, pendulum swinging world of software development good things are abandoned when no longer novel and are exaggerated to the point that they become harmful. It’s not a world known for thoughtful reflection and consideration. But saying that pull request reviews undermine trust and cohesion in teams or a formulaic practice without underlying benefit seems unhelpfully controversial and doctrinal.

Standard
Programming

Recommendation: use explicit type names

I attended an excellent talk recently at Lambdale about teaching children to code (not just Scratch but actual computer languages) and one of the points of feedback was that the children found the use of names in type declarations and their corresponding implementations in languages such as Haskell confusing because frequently names are reused despite the two things being completely different categories of things.

The suggestion was that instead of f[A] -> B it was simpler to say f[AType] -> BType.

Since I am in the process of introducing Python type checking at work this immediately sparked some interest in me and when I returned to work I ran a quick survey with the team that revealed that they too preferred the explicit type name with the suffix Type rather than package qualifier for types that I had been using.

Instead of kitchen_types.Kettle the preference was for KettleType.

The corresponding function type annotations would therefore read approximately:

def boil(water: WaterType, kettle: KettleType) -> WaterType

So that’s the format we’re going to be adopting.

Standard
Programming

HyLang: Sorted

Sorted in Hylang reports as being a built-in but isn’t documented in the HyLang built-in documentation. After a bit of head-scratching I realised that this is just the Python built-in sorted.

Hy uses keyword parameters to bind keyword arguments in Python interop so to define a new sorting function you assign a lambda to the key keyword.

So to sort pairs by the second value…

(sorted :key (fn [[a b]] b) items)

This is pretty similar to Clojure’s sort-by except that you can’t define the comparator explicitly.

Standard
Programming

After having checked our privilege, how shall we continue?

I’ve been privileged enough to go to a number of development conferences last year (2018). I know that I’m privileged because almost every conference had a talk or many talks making clear my privilege in attending them.

I have a job where attending conferences makes sense as part of my work and I make enough money to be able to buy my own tickets and pay for my travel and accommodation. I’m often able to combine attending a conference with catching up with friends and family. I have the ability to choose which conferences I attend based on whether they allow me to achieve other things I value in my life.

I’m lucky and well-placed in life, and if I was unaware of that then fortunately every conference will have a speaker willing to point that out to me. Sometimes for as long as an hour.

An hour that I’m not unaware that I have paid a lot of money for, an hour that I have chosen to spend listening to this talk instead of doing something that I might enjoy instead.

Often these conferences talks have no particular point they want to make beside how privileged people in tech are and how little we understand people outside our tech bubble. They have no clear or sensible strategy as to how to change what they regard as disagreeable.

Often they feel very unclear about what exactly is wrong with the privilege enjoyed by their audience. Perhaps eliminating it a worthy goal in itself.

Its certainly not clear what audience the speakers would be happy to address about the topics that might be related to the notional agenda of subject of the conference they are speaking at.

The vagaries of conference programming committee means that there will often be another talk taking about how difficult it is to be a programmer: how prone to stress and burnout we are and how we need to prioritise self-care.

We are self-absorbed and toxic, while also being fragile and in need of nurture. No talk has addressed this contradiction.

Topics such as privilege and the self-absorption of tech are potentially worthy subjects but at the end of a year, having heard variations of these talks many, many times I want to stop.

I’m happy to nurture my privilege checking and think about the technology needs of the emerging world while taking steps to create an inclusive workforce.

In return, what I would like from the conferences that I pay to attend is some attempt to deliver a programme that reflects the prospectus that is laid out when you buy a ticket to say a Javascript conference or a Python conference.

The minimum I would like to have is that in every timetable slot there is a strong technology talk that will be relevant to my work and interests, preferably something informative and provided by a technology practitioner.

If this can happen then I’ll feel as if attending the conference was a good thing in itself rather than the peg on which to hang the chance to visit places and people.

Standard
Programming

Tackley’s law of caching

Tackley’s law of caching is that the right cache time is the time it takes someone to come to your desk to complain about a problem.

If you obey this law then when you open your browser to check the issue the person is complaining about it will already have resolved itself.

Tackers may not have invented this rule but I heard it from him first and it one of the soundest pieces of advice in software development I’ve ever had.

Standard
Programming

Slow SPAs are worse than NoSPA

I got a digital subscription to the Economist for my birthday last month so I’ve started reading a lot more content on their site. As a result I’ve noticed a lot of weirdness with their page loads that was hardly noticeable when I was using the free tier of a few articles per week.

The site seems to be built as a SPA with a page shell that loads quite quickly but takes far longer to fill with content and which has some odd layout choices and occasional pops and content shifts.

The basic navigation between the current issue index and the articles is hampered by what appears to be a slow load or render phase. Essentially it is hard to know whether the click on a link or the back button has registered.

By replacing traditional page navigation the experience is actually worse. The site would be better if the effort going into the frontend went into faster page serving.

I’m not sure if the page is meant to be doing something clever with local storage for offline use but it seems to need to be connected when browsing so I’m assuming that this is something to do with the need for a subscription and payment gateway that prevents a fast server load of content.

It still feels as if the page and the 200 words or so should be public and CDN-cached with the remaining content of the article being loaded after page-load for subscribers.

The current solution feels like someone has put a lot of effort and thought into making someone that is actually worse than a conventional webpage and that seems a shame for a site with relatively little content that is mostly updated once a week.

Standard