Author Archives: rrees

Month notes

January 2025 month notes

Another relatively quiet month. I’ve mostly been consolidating my tooling around just, uv and Node 23 (see my post on type stripping).

I did try to start switching from Prettier to Biome but it hasn’t all been smooth. I think I need to remove the Prettier plugins from my editors and I had to do some manual setting tweaking to get auto-formatting on save. I’ve increasing being preferring Ruff to Black, UV to pipenv and I suspect it isn’t going to be much different once I get Biome working.

I also tried using Claude for creating specific bits of code and it was a lot better than the Github Copilot experience. I think was the first time that I was getting better code than I could have written as interestingly some of the problems I was asking for solutions to had a wider range of inputs than I was considering. In my particular usecase these categories of input wouldn’t have occurred but I didn’t have any good reason as to why they would be excluded so my own code would probably have failed in some reasonably common scenarios.

Still I feel that so far I’ve been asking for solutions to problems that I know have been solved by someone, I just don’t know where to find the code. I haven’t tried to do anything properly hard or novel yet.

Standard
Programming

Type stripping in Node

I tried the new type stripping feature which is enabled by default in Node 23.6.0. Essentially the type information is stripped from the code file and things that require transformation are not supported by default yet. You’re essentially writing Typescript and running Javascript. This is essence of the Typescript promise that there’s nothing in the language that isn’t Javascript (ignore enums).

On the surface of it this might seem pointless but the genius of it is that all the source files are Typescript and all tooling works as normal. There’s just no complicated build or transformation step. I have some personal projects that were written as Javascript but which were using typing information via JSDoc. Type stripping is much less complicated and the type-checking and tooling integration is much richer.

I’m planning to try and convert my projects and see if I hit any issues but I feel that this should be the default way to use Typescript now and that it is probably worth looking to try and upgrade older projects. The gulf of functionality between Node 18 and 23 is massive now.

Standard
Events

Barcamp 13

Barcamp 13 is a general tech and nerdery unconference in London. In its current incarnation it is a one day event at an academy school next to the Tottenham Hotspur stadium.

Although most of the topics were technology related the analog sessions were amongst the most memorable, in particular the Cèilidh session was really fun and a total change in energy. I wasn’t expecting to dance when arrived. I also enjoyed the Minimal Viable Zine session which was about making single sheet zines for communicating urgent information indirectly but person to person.

The work relevant sessions included Nested CSS, which I’ve now started to adopt for my CSS work and looking at how to apply retrospectively to my hobby projects.

I also went to a session about tackling polarisation which was quite interesting as it had both self-proclaimed leftists and Hungarian fans of Orban. I was curious as to why the convener felt that polarisation was new and a problem. The answer the group came up with was that if the polarisation results in shrinking the envelope of who are considered people or a reduction in the rights of people in society then you are potentially talking about life and death. We’ve seen this in the treatment of both refugees and trans rights.

A few things that came through in this session was that there was a strong belief in the power of media (traditional and social) to change social attitudes. I think that would be something interesting to follow up on.

There was also a session on AI-generated music which was interesting but also worrying and which I think probably deserves a post in its own right.

I learned some things, I had fun, I enjoyed chatting to the other attendees and I managed to not be on fire. It was a super interesting day and I will definitely make an effort to get to the next one.

Standard
Month notes

December 2024 month notes

Not a whole lot to report on due to this being holiday season.

Colab

I started using Google’s Colab for quick Python notebooks. It’s pretty good and the notebook files integrate into regular Drive. Of course there is always the fear that Google will cancel it at a moment’s notice so I might look at the independent alternatives as well.

I’ve been looking at simulation code recently and it has been handy to run things outside a local setup and across laptops.

tRPC

You can’t spend much time in Typescript world without using Zod somewhere in your codebase. Zod was created by Colin McDonnell and this month I read an old blog post of his introducing the ideas behind tRPC. The post or essay is really interesting as it identifies a lot of problems that I’ve seen with GraphQL usage in projects (and to be fair some OpenAPI generated code as well).

It is quite rare to see a genuine REST API in the commercial world, it is more typical to see a REST and HTTP influenced one. GraphQL insists on a separation of the concepts of read (query) and write (mutation) which makes it more consistent than most REST-like interfaces but it completely fails to make use of HTTP’s rich semantics which leaves things like error handling as a bit of joke.

Remote Procedure Calls (RPC) preceded both REST and GraphQL and while the custom protocols and stub generators were dreadful the mental model associated with RPC is actually pretty close to what most developers actually do with both REST and GraphQL. They execute a procedure and get a return result.

Most commercial-world REST APIs are actually a kind of RPC over HTTP using JSON. See the aside in the post about GraphQL being RPC with a schema.

Therefore the fundamental proposition of the post seems pretty sound.

The second strong insight is that sharing type definitions is far preferable and less painful than sharing generated code or creating interface code from external API definitions (I shudder when I see a comment in a codebase that says something like “the API must be running before you build this code”). This is a powerful insight but one that doesn’t have a totally clean answer in the framework.

Instead the code reaches out to import the type definition from the server by having the local codebase available in some agreed location. I do think this is better than scraping a live API and type sharing code is clearly less coupled than sharing data structures but I’m not sure it is quite the panacea being claimed.

What it undoubtedly does improve on is generated code, generated code is notorious hard to read, leads to arguments about whether it should be version controlled or not and when it goes wrong there is almost inevitably the comparison dance between developers who have working generated code and those who don’t. Having a type definition that is version controlled and located in one place is clearly a big improvement.

I’ve only seen a few mentions of commercial use of tRPC and I haven’t used it myself. It is a relatively small obscure project but I’d be interested in reading production experience reports because on the face of it it does seem to be a considered improvement over pseudo-REST and GraphQL interfaces.

God-interfaces

The article did also remind me of a practice that I feel might be an anti-pattern but which I haven’t had enough experience so far to say for sure. That is taking a generated type of a API output and using it as the data type throughout the client app. This is superficially appealing: it is one consistent definition shared across all the code!

There are generally two problems I see with this approach, firstly is protocol cruft (which seems to be more of a problem with GraphQL and automagic serialisation tools) which is really just a form of leaky abstraction; secondly, if a data type is a response structure from a query type of endpoint then the response often has a mass of optional fields that continuously accrue as new requirements arrive.

You might be working on a simple component to do a nicely formatted presentation of a numeric value but what you’re being passed are twenty plus fields, none of which might exist or have complex dependencies between one another.

What I’ve started doing, and obviously prefer, is to try and isolate the “full fat” API response at the root component or a companion service object. Every other component in the client should use a domain typed definition of its interface.

Ideally the naming of the structures in the API response and the client components would allow each domain interface to be a subset of the full response (or responses) if the component is used across different endpoints.

In Typescript terms this means components effectively define interfaces for their parameters and passing the full response object to the component works but the code only needs to describe the data actually being used.

My experience is that this has led to code that is easier to understand, is easier to modify and is less prone to breaking if the definition of the API response changes.

The death of the developer

I’ve been reading this Steve Yegge post a lot as well The Death of the Stubborn Developer. Steve’s historical analysis has generally been right which gives me a lot of pause for thought in this post. He’s obviously quite invested in the technology that underpins this style of development though and I have a worry that it is the same kind of sales hustle that was involved in crypto. If people don’t adopt this then how is the investment in this kind of assisted coding going to be recouped.

Part of what I enjoy about coding is the element of craft involved in putting together a program and I’m not sure that the kind of programming described in the post is the kind of thing I would enjoy doing and that’s quite a big thing given that it has been how I’ve made a living up until now.

Standard
Programming

Notes on calling Rust from Javascript

Although the tutorial looks great and comprehensive it describes a process that is surprisingly cumbersome. If you switch to the modern web approach things seem to get a lot simpler. This simply requires you to set the target of the output to be web.

With this done and a little bit of boilerplate (which I need to extract and make reusable somehow) which consisted of a static page, a one-line dynamic import Javascript file (which I’m not sure is needed) and a variant of the generated bootstrap file I was able to write any pure function using numbers or strings in Rust and then use it in short-order in the static page.

wasm-pack outputs a lot of files, there is the WASM binary itself which seems straight-forward, a couple of Typescript definition files which I presume is to help integrate the output into Typescript projects if you’re shipping the output as a module. There are two Javascript files that seem to define the Javascript version of the Rust functions and the value conversion between the two languages. I’m not sure why there are two, I might need to do a clean build and see what comes out. Maybe the smaller Javascript file actually contains everything needed for the web.

The way things feel at the moment is that if you’re stuck with a build issue the errors and the solution can feel incomprehensible but once it works it works really well and it is shockingly easy to export callable functions.

Apart from trying to find the minimum viable build I also need to try and implement something that is a bit more computationally challenging in the Rust function and get a sense of what the overhead of the Web Assembly call is because it feels relatively minimal.

Standard
Software

Thoughts on the ethical use of LLMs

Large language models (LLMs) have felt fraught with issues. Let’s start with the environmental impact which has been completely disastrous and has essentially led to Big Tech ditching all their Net Zero promises. Vast amounts of speculative money have led to truly insane amounts of energy being spent creating models that don’t have strongly differentiated capabilities. Between this and cryptocurrency you’d be surprised to discover that electricity is not free and actually has consequences to its use.

Then there is the question of the corpus, mass ingestion of content for training AIs combined with obfuscation on the original of that material has resulted in a toxic situation for people who feel they have been taken advantage of and dubious legal situation for people using the output of such models.

The inherent flaws of the models’ probabilistic nature (hallucination, non-determinism) combined with user’s flawed mental models of what is happening is causing all kinds of strange fallout.

Finally there are the way that LLMs are being applied to problems, namely without any discretion or thought as to whether they have any relevance to the situation in hand. Again that glut of money at a time when most businesses are being squeezed by interest rates means that what gets used is what funders are excited about not what users need.

Now I’m not anti-AI or LLM in principle. I think there are some strong use-cases: summarisation, broad textual or structured document analysis and light personalisation. All machine models have infinite patience for user interaction and it seems humans prefer the tone of model generated content to that created by humans (which creates the burning question, why?) (2025-01-16: this article on how cognitive biases feed into interpretations of chat bot interactions seems relevant but it also includes an important reminder that the models are human ranked and tuned before they are released so I think it is natural that high agreeability would score well and unfriendly models would be binned). I think LLMs with access to vast amounts of information help put a floor under people’s understanding of problems and how to tackle things which is why individual subscriptions have been more popular than institutional ones.

However the foundation under these valid use cases needs to be sound and it currently it isn’t.

The new models by Pleais show that it is possible to do a better job of these problems. By having clearer information about the provenance of the information and the terms under which the team were allowed to use it. They have also been open about the carbon cost of training the model.

There still remain questions about the carbon cost of running the model and some about what the researchers mean about generating additional material for their corpus but this feels like the minimum that the bigger players should be offering.

The clarity over the training set should help alleviate the concerns people have about content being exploited with componsation or permission. Clear carbon figures mean we can start to compare the cost of creating new models and start to measure the efficiency of such efforts. Such a consideration should maybe be a factor in deciding whether a training process should be continued or not.

Privacy concerns can be alleviated by running models locally as well as insisting on greater clarity in the terms of service of the cloud providers (something I think Amazon moved closer towards with their Nova models).

I believe it is possible to address the genuine concerns people have about LLMs and to benefit from their use but the problems need to be acknowledge and addressed in a way that the mad scramble for AI gold simply has not done so far.

Standard
Software

Halfstack London 2024

Halfstack is a really interesting conference which is increasingly heading into a space that is unrepentantly about hobby projects, the love of technology and amateurism. But also a few talks by professional developer advocates or relationship managers.

It happens on Brick Lane and has an open bar in the afternoon. You might think that this is either hip or insufferable. Both opinions might be right.

In terms of work relevant material there was an interesting if often incoherent talk on the phases of the event loop which seems to be a popular topic but which was full of new information for me (being pretty basic I suppose); a sales pitch for the developer experience in Deno and using Tensorflow JS to do image recognition in mode.

Christian Heilmann has switched from giving talks on developer tools in the browser to the state of developer employment and this time highlighted the dilemma facing junior developer roles. While demand has fallen back for developers (compared to the incredible growth in demand for the previous five to ten years) it has done dramatically for entry-level roles and less so for experienced developers.

When the industry turns off the pipeline like this the effects tend to take years to feed through, as experienced people retire or switch to other roles there are less people taking their place as entrants have responded to the market signal and are doing something else.

The industry gamble here is that AI is going to make up the gap but the risk is whether the people using the AI are going to have a deep enough understanding of what is being created that they can support and maintain the result.

Maintaining codebases was in a way the theme of the talk, with all the emphasis on producing more code with the help of AIs is anyone thinking about what will happen to these digital products in five years time?

The real highlight of the day was a talk that combined the history of the 808 and 909 with a reminder of how crazy some of the browser API support is. Did you know that your browser probably supports MIDI?

According to the talk, you can read the slides online, the 808 and 909 were both flops on release that became classics after hitting the bargain bin so that a different kind of musician could access them and apply a different aesthetic sense to their capabilities.

The talk then used web APIs to recreate the 808 sound with samples via the ToneJS library and to trigger them with a USB connected device (less well supported). That was followed up with a mini-sequencer that was good enough to do a little live performance.

The day ended with a talk on using technology in murder mystery parties which was a bit crazy and obsessional and interesting in the way that people who have Gone Too Far can be. There was a bit where a trunk was being wired up to the mains where I thought the biggest danger might be the risk of death by homemade electronics.

Tickets for 2025 are available and next year’s conference and it has just recently been confirmed that enough pre-sales have been made to ensure the venue can be booked and that therefore next year’s conference is definite. In a period of declining sponsorship and stretched personal budgets that’s a vote of confidence in the conference from its audience.

Standard
Month notes

November 2024 month notes

Rust tools

Rust seems to be becoming the defacto standard for tooling, regardless of the language being used at a domain level. This month I’ve talked to people from Deno who build their CLI with it, switched to the just command runner and ruff code formatter.

It’s an interesting trend in terms of both other languages being more comfortable about not writing their tooling in a different language and why Rust seems to have a strong showing in this area.

Gitlab pipelines

I have been working a lot with Gitlab CI/CD this month, my first real exposure to it. Some aspects are similar to Github Actions, you’re writing shell script in YAML and debugging is hard.

Some of the choices in the Gitlab job environments seems to make things harder than they need to be. By default the job checks out the commit hash of the push that triggered the build in a detached (fetch) mode. Depending on the natural of the commit (in a merge request, to a branch, to the default (main) branch) you seem to get different sets of environment variables populated. Choose the wrong type and things just don’t work, hurrah!

I’ve started using yq as tool for helping validate YAML files but I’m not sure if there is a better structural tool or linter for the specific Gitlab syntax.

Poetry

I’ve also being doing some work with Poetry. As everyone has said the resolution and download process is quite slow and there doesn’t seem to be a huge community around it is a tool. Its partial integration with pyproject.toml makes it feel more standard that it actually is with things under the Poetry key requiring a bit of fiddling to be accessible to other tools. Full integration with the later standard is expected in v2.

Nothing I’ve seen so far is convincing me that it can really make it in its current form. The fragmentation between the pure Python tools seems to have taken its toll and each one (I’ve typically used pipenv) has problems that they struggle to solve.

RSS Feeds

One of the best pieces of advice I was given about the Fediverse was that you need to keep following people until your timeline fills up with interesting things. I’ve been trying to apply that advice to programmers. Every time I read an interesting post I’m now trying to subscribe. Despite probably tripling the number of feeds I have subscribed to my unread view is improved but still dominated by “tech journalism”. I guess real developers probably don’t post that frequently.

Lobsters has been really useful for highlighting some really good writers.

CSS

Things continue to be exciting in the CSS world with more and more new modules entering into mainstream distribution (although only having three browsers in the world is probably helping). I had a little play around with Nested Selectors and while I don’t do lots of pseudo-selectors it is 100% a nice syntax for them. In terms of scoping rules, these actually seem a bit complex but at least they are providing some modularity. I think I’m going to need to play more to get an opinion.

The Chrome developer relations team have posted their review of 2024.

Not only is CSS improving that but Tailwind v4 is actually going to support (or improve support) some of these new features such as containers. And of course its underlying CSS tool is going to be Rust-powered, natch.

Standard
Month notes

October 2024 month notes

For small notes, links and thoughts see my Prose blog.

Web Components versus frameworks

Internet drama erupted over Web Components in what felt a needless way. Out of what often felt wasted effort there were some good insights Lea Verou had a good overview of the situation, along with an excellent line about standards work being “product work on hard mode”

Chris Ferdinandi had a good response talking about how web components and reactive frameworks can be used together in a way that emphasises their strengths.

One of my favourite takes on the situation was by Cory LaViska who pointed out that framework designers are perhaps not the best people to declare the future of the platform.

Web Components are a threat to the peaceful, proprietary way of life for frameworks that have amassed millions of users — the majority of web developers.

His call to iterate on the standard and try to have common parts to today’s competing implementations was echoed in Lea’s post.

The huge benefit of Web Components is interoperability: you write it once, it works forever, and you can use it with any framework (or none at all). It makes no sense to fragment efforts to reimplement e.g. tabs or a rating widget separately for each framework-specific silo, it is simply duplicated busywork.

The current Balkanisation of component frameworks is really annoying and it is developer’s fear and tribalism that has allowed it to happen and which has sustained it.

Postgres generated UUIDs

In my work I’ve often seen UUIDs be generated in the application layer and pushed into the database. I tried this in a hobby project this month and rapidly came to the conclusion that it is very tedious when you can just have the database handle it. In Postgres a generated UUID can just be the column default and I don’t think I’m going to do anything else in future if I have a choice about it.

Python 3.13

I’ve started converting my projects to the new Python version and it seems really fast and snappy even on projects that have a lazy container spin-up. I haven’t done any objective benchmarking but things just feel more responsive than 3.11.

I’m going to have a push to set this as the baseline for all my Python projects. For my Fly projects extracting out the Python version number as a Docker variable has meant migrating has been as simple as switching the version number so far.

For the local projects I’ve also been trying to use asdf for tool versioning more consistently and it has made upgrading easier where I’ve adopted it but it seems I have quite a few places where I still need to convert from either language specific tools or nothing.

uvx

uvx is part of the uv project and I started using it this month and its rapidly becoming my default way to run Python CLIs. The first thing I started using it with was pg-cli but I found myself using it to quickly run pytest over some quick scripting code I’d done as well as running ad-hoc formatters and tools. It’s quick and really handy.

There’s still the debate about whether the Python community should go all-in on uv, looking at the messy situation in Node where all manner of build and packaging tools could potentially be used (despite the ubiquity of npm) the argument for having a single way to package and run things is strong.

Standard
Blogging

Python x Rust meetup October 2024

This was the inaugural (hopefully) Rust/Python meetup crossover and to be honest it was heavy on the Python developers incorporating Rust into their code.

The talks were mostly recycled from other contexts (but no worse for having a bit of polish): David Seddon Crabs in Snakes!, David Hewitt How Python harnesses Rust, Samuel Colvin Pydantic v2 with Rust.

If pushed for time I would probably recommend David Hewitt’s one if you want to learn about how PyO3 (and foreign calls in Python generally) works and David Seddon’s if you just want to try and get a working Python-Rust interop.

During the Q&A Samuel made probably the key point of the evening for me that virtually anything you do in Rust is going to be faster than Python (and for good reasons, this isn’t a diss on Python). The call overhead is negligible and you’ll be using a system language. His case in point was string validation, even relatively simple validation is faster in Rust. David Hewitt’s example was Array processing versus Python’s dynamic Lists.

The disadvantages that there were mentioned about Rust was its complexity, which is simplified when the context is a function call and it being hard to try out different ideas and reorganise code, the strict compiler means that you need to move from one complete implementation to another.

If interop becomes increasing seamless (and it sounds like the biggest problem currently is having to build binaries) then there will be no reason not to drop from Python into Rust. There also won’t be a strong reason for not starting your programming in a high-level abstraction and dropping down a level when you need to.

Standard