Tag Archives: web development

Programming

Type stripping in Node

I tried the new type stripping feature which is enabled by default in Node 23.6.0. Essentially the type information is stripped from the code file and things that require transformation are not supported by default yet. You’re essentially writing Typescript and running Javascript. This is essence of the Typescript promise that there’s nothing in the language that isn’t Javascript (ignore enums).

On the surface of it this might seem pointless but the genius of it is that all the source files are Typescript and all tooling works as normal. There’s just no complicated build or transformation step. I have some personal projects that were written as Javascript but which were using typing information via JSDoc. Type stripping is much less complicated and the type-checking and tooling integration is much richer.

I’m planning to try and convert my projects and see if I hit any issues but I feel that this should be the default way to use Typescript now and that it is probably worth looking to try and upgrade older projects. The gulf of functionality between Node 18 and 23 is massive now.

Standard
Month notes

November 2024 month notes

Rust tools

Rust seems to be becoming the defacto standard for tooling, regardless of the language being used at a domain level. This month I’ve talked to people from Deno who build their CLI with it, switched to the just command runner and ruff code formatter.

It’s an interesting trend in terms of both other languages being more comfortable about not writing their tooling in a different language and why Rust seems to have a strong showing in this area.

Gitlab pipelines

I have been working a lot with Gitlab CI/CD this month, my first real exposure to it. Some aspects are similar to Github Actions, you’re writing shell script in YAML and debugging is hard.

Some of the choices in the Gitlab job environments seems to make things harder than they need to be. By default the job checks out the commit hash of the push that triggered the build in a detached (fetch) mode. Depending on the natural of the commit (in a merge request, to a branch, to the default (main) branch) you seem to get different sets of environment variables populated. Choose the wrong type and things just don’t work, hurrah!

I’ve started using yq as tool for helping validate YAML files but I’m not sure if there is a better structural tool or linter for the specific Gitlab syntax.

Poetry

I’ve also being doing some work with Poetry. As everyone has said the resolution and download process is quite slow and there doesn’t seem to be a huge community around it is a tool. Its partial integration with pyproject.toml makes it feel more standard that it actually is with things under the Poetry key requiring a bit of fiddling to be accessible to other tools. Full integration with the later standard is expected in v2.

Nothing I’ve seen so far is convincing me that it can really make it in its current form. The fragmentation between the pure Python tools seems to have taken its toll and each one (I’ve typically used pipenv) has problems that they struggle to solve.

RSS Feeds

One of the best pieces of advice I was given about the Fediverse was that you need to keep following people until your timeline fills up with interesting things. I’ve been trying to apply that advice to programmers. Every time I read an interesting post I’m now trying to subscribe. Despite probably tripling the number of feeds I have subscribed to my unread view is improved but still dominated by “tech journalism”. I guess real developers probably don’t post that frequently.

Lobsters has been really useful for highlighting some really good writers.

CSS

Things continue to be exciting in the CSS world with more and more new modules entering into mainstream distribution (although only having three browsers in the world is probably helping). I had a little play around with Nested Selectors and while I don’t do lots of pseudo-selectors it is 100% a nice syntax for them. In terms of scoping rules, these actually seem a bit complex but at least they are providing some modularity. I think I’m going to need to play more to get an opinion.

The Chrome developer relations team have posted their review of 2024.

Not only is CSS improving that but Tailwind v4 is actually going to support (or improve support) some of these new features such as containers. And of course its underlying CSS tool is going to be Rust-powered, natch.

Standard
Month notes

October 2024 month notes

For small notes, links and thoughts see my Prose blog.

Web Components versus frameworks

Internet drama erupted over Web Components in what felt a needless way. Out of what often felt wasted effort there were some good insights Lea Verou had a good overview of the situation, along with an excellent line about standards work being “product work on hard mode”

Chris Ferdinandi had a good response talking about how web components and reactive frameworks can be used together in a way that emphasises their strengths.

One of my favourite takes on the situation was by Cory LaViska who pointed out that framework designers are perhaps not the best people to declare the future of the platform.

Web Components are a threat to the peaceful, proprietary way of life for frameworks that have amassed millions of users — the majority of web developers.

His call to iterate on the standard and try to have common parts to today’s competing implementations was echoed in Lea’s post.

The huge benefit of Web Components is interoperability: you write it once, it works forever, and you can use it with any framework (or none at all). It makes no sense to fragment efforts to reimplement e.g. tabs or a rating widget separately for each framework-specific silo, it is simply duplicated busywork.

The current Balkanisation of component frameworks is really annoying and it is developer’s fear and tribalism that has allowed it to happen and which has sustained it.

Postgres generated UUIDs

In my work I’ve often seen UUIDs be generated in the application layer and pushed into the database. I tried this in a hobby project this month and rapidly came to the conclusion that it is very tedious when you can just have the database handle it. In Postgres a generated UUID can just be the column default and I don’t think I’m going to do anything else in future if I have a choice about it.

Python 3.13

I’ve started converting my projects to the new Python version and it seems really fast and snappy even on projects that have a lazy container spin-up. I haven’t done any objective benchmarking but things just feel more responsive than 3.11.

I’m going to have a push to set this as the baseline for all my Python projects. For my Fly projects extracting out the Python version number as a Docker variable has meant migrating has been as simple as switching the version number so far.

For the local projects I’ve also been trying to use asdf for tool versioning more consistently and it has made upgrading easier where I’ve adopted it but it seems I have quite a few places where I still need to convert from either language specific tools or nothing.

uvx

uvx is part of the uv project and I started using it this month and its rapidly becoming my default way to run Python CLIs. The first thing I started using it with was pg-cli but I found myself using it to quickly run pytest over some quick scripting code I’d done as well as running ad-hoc formatters and tools. It’s quick and really handy.

There’s still the debate about whether the Python community should go all-in on uv, looking at the messy situation in Node where all manner of build and packaging tools could potentially be used (despite the ubiquity of npm) the argument for having a single way to package and run things is strong.

Standard
Work

January 2024 month notes

Water CSS

I started giving this minimal element template a go after years of using various versions of Bootstrap. It is substantially lighter in terms of the components it offers with probably the navigation bar being the one component that I definitely miss. The basic forms and typography are proving fine for prototyping basic applications though.

Node test runner

Node now has a default test runner and testing framework. I’ve been eager to give it a go as I’ve heard that it is both fast and lightweight, avoiding the need to select and include libraries for testing, mocking and assertions. I got the chance to introduce it in a project that didn’t have any tests and I thought it was pretty good although it’s default text output felt a little unusual and the alternative dot notation might be a bit more familiar.

It’s interesting to see that the basic unit of testing is the assertion, something is shares with Go. It also doesn’t support parameterised tests which again is like Go which has a pattern of table-driven tests implemented with for loops except that Go allows more control of the dynamic test case naming.

I’d previously moved to the Ava library and I’m not sure there is a good reason not to use the built-in alternative.

Flask blueprints

In my personal projects I’ve tended to use quite a few cut and paste modules and over the years they tend to drift and get out of sync so I’ve been making a conscious effort to learn about and start adopting Flask Blueprints. Ultimately I want to try and turn these into personal module dependencies that I can update once and use in all the projects. For the moment though it is interesting how the blueprints format is pushing me to do some things like logging better (to understand what is happening in the blueprint) and also structuring the different areas of the application so that they are quite close to Django apps with various pieces of functionality now starting to be associated with a url prefix that makes it a bit easier to create middleware that is registered as part of the Blueprint rather than relying on imports and decorators.

Web components

I’ve been making a bit of progress with learning about web components. I realised that I was trying to do too much initially which is why they were proving complicated. Breaking things down a bit has helped with an initial focus on event listeners within the component. I’m also not bringing in external libraries at the moment but have got as far as breaking things up into [ESM modules](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules) which has mostly worked out so far.

Standard
Programming

Halfstack on the Shore(ditch) 2023

Self-describing as an “anti-conference” or the conference that you get when you take all the annoying things about conferences away. It is probably one of the most enjoyable conferences I attend on a regular basis. This year is was in a new venue quite close to the previous base at Cafe 1001 which was probably one of my favourite locations for a conference.

The new venue is a small music venue and the iron pillars that fill the room were awkward for sightlines until I grabbed a seat at the front. The bar opened at midday and was entirely reasonable but the food was not as easily available as previously available but you were still able to walk to the nearby cafe and show your conference badge if you wanted.

Practical learnings

Normally I would say that HalfStack is about the crazy emergent stuff so I was surprised to actually learn a few things that are relevant to the day job (admittedly I have been doing a lot more backend Javascript than I was previously). I was quite intrigued to see some real-world stats that the Node’s in-built test runner is massively faster than Jest (which maybe should not be so surprising as it does some crazy things). I’ve been using [Bun]() recently which does have a faster runner and it makes TDD a lot more fun that with the normal Jest test runner.

I also learnt that NODE_ENV is used by library code to conditionally switch on paths in their code. This is obviously not a sound practice but the practical advice was to drop variables that map to environments completely and instead set parameters individually as per standard 12 factor app practice. I think you can refine that with things like dotenv but I’m basically in agreement. Two days later I saw a bunch of environment-based conditional code in my own workplace source code.

It was also interesting to see how people are tackling their dependency testing. It felt like the message is that your web framework should come with mocks or stubs for testing routing and requests as standard and that if it doesn’t then maybe you should change your framework. That feels a bit bold but that’s only because Javascript is notorious for having anaemic frameworks that offer choice but instead deliver complexity and non-trivial decisions. On reflection it seems like having a built-in unit testing strategy for your web framework seems like a must-have feature.

Crazy stuff

There was definitely less crazy stuff than in previous years. A working point of sale system including till management based on browser APIs was all quite practical and quite a good example of why you might want USB and serial port access within the browser.

There was also a good talk about converting ActionScript/Flash to Javascript and running emulation of old web games although that ultimately turned out to be a way of making a living as commercial games companies wanted to convert their historic libraries into something that people could continue to use rather than being locked away in a obsolete technology.

The impact of AI

One of the speakers talked about using ChatGPT for designing pitches (the generated art included some interesting interpretations our how cat claws work and how many claws they have) and I realised listening to it that for some younger people the distilled advice and recommendations that the model has been fed is exactly the kind of mentoring that they have desired. From a negative perspective this means an endless supply of non-critical ideas and suggestions that require little effort on the user’s part; just another way to avoid having to do some of the hard work of deliberative practice. On the positive side a wealth of knowledge that is now available to the young in minutes.

While I might find the LLMs trite, for people starting their careers the advice offered is probably more sound that their own instincts. There also seems to be some evidence appearing that LLMs can put a floor under poor performance by correctly picking up common mistakes and errors. At a basic level they are much better at spelling and grammar than non-native speakers for example. I don’t think they have been around long enough to have reliable information though and we need to decide what basic performance of tasks looks like.

I wonder what the impact will be on future conference talks as ChatGPT refines people to a common set of ideas, aesthetics and structures. Probably it will feel very samey and there will be a desire to have more quirky individual ideas. It feels like a classic pendulum swing.

Big tech, big failings

Christian Heilmann’s talk was coruscating about the failures of big tech during the acute phase of the COVID pandemic and more generally about being unable to tackle the big problems facing humanity and instead preferring to focus on fighting for the attention economy and hockey stick growth that isn’t sustained. He also talked about trying to persuade people that they don’t have to work at FAANGS to be valid people in technology.

His notes for this talk are on his blog.

Final thoughts

Chat GPT might need me to title this section as a conclusion to avoid it recommending that I add a conclusion. HalfStack this year is happening at a strange time for programming and the industry. There wasn’t much discussion of some topics that would have been interesting around the NodeJS ecosystem such as alternative runtimes and the role of companies, consultancy and investment money in the evolution of that ecosystem. The impact of a changed economic environment was clear and in some cases searing but it was a helpful reminder that it is possible to find your niche and make a living from it. You don’t necessarily need to hustle and try and make it big unless that is what you really want to do.

The relaxed anti-conference vibe felt like a welcome break from the churn, chaos and hamster wheel turning that 2023 has felt like. I’ve already picked up my tickets for next year.

Links

Standard
Programming

State of the Browser 2022

I’ve attended a few of these conferences and have always found them helpful. This year it had relocated to the Barbican Centre with the food and drink area overlooking the beautiful Conservatory there, great choice as a venue.

The conference was a hybrid in-person/online event that I think could serve as a model for other conferences that seem to only be focusing on their return to in-person. Due to other commitments I wasn’t able to be at the venue all day and so at lunchtime I headed home and picked up a few of the rest of the talks on the livestream. It was great to have the flexibility and made the whole conference more accessible.

Talks-wise it was interesting as ever and a little bit less inward looking or niche interest that it has been in the past. There were the usual mix of upcoming standards and challenges in implementing them, how to apply techniques to the current broad mainstream of browsers and a little bit of evangelism for playfulness and environment impact.

One of my key takeaways was on this last point; using an image CDN that can do automatic content negotiation to use an efficient modern image standard has a huge carbon saving. It feels a bit crazy that so many companies are still serving fixed sizes and formats off things like Cloudfront and S3.

Bruce Lawson kicked off the event with a good historical perspective talk on the history of standards (and the struggle to create and maintain them) and brought the issues of standardisation through the search for technical solutions to the world of regulation and better digital policy. Engaging with law makers is a more realistic way to improve the online world that the search of technical solutions to social problems.

More practically we can hope that Apple will be compelled as a digital gatekeeper to allow competition on browser implementations on its platform and maybe even fund its Safari team properly to have better compatibility with the general web standards on iOS. I felt it was nice for a recognition that government organisations can be engaged and willing to listen and that progress can be made be working together rather than outside of regular power structures.

Probably the best talk I heard was “Be the browser’s mentor not it’s micromanager” by Andy Bell this talk neatly encompassed two major ideas: the first was the way that layout systems in CSS have advanced to the point where you are describing structure and allowing the layout manager to actually decide the rendering and secondly on how digital design approaches have managed to fall between the abstractions of the grid system and the precise layout of magazine style layout.

By leaning on the layout engines the amount of CSS we have to write is much more minimal than the micromanaging fussiness typical to component design systems. It is also more powerful and expressive, avoiding the overly complex muddle that is often associated with component style systems but also not going too far down the class frenzy of utility class systems.

Sophie Koonin taught me how to use the prefers-reduced-motion preference via the medium of late 90s website chaos. A good example of the mixture of fun and practical content.

I also enjoyed Alistair Shepherd‘s talk which had a few technical bits and pieces but managed to bridge the themes of the conference by wanting to create a personal website that first and foremost reflected his personal interests and then used the tech to deliver the vision he had for himself. Although the idea to have websites that vary according to the time of date is quite an interesting idea.

I didn’t catch the last few talks so I’m hoping to be able to watch them when they come to YouTube (or maybe some federated alternative!).

Overall still one of the necessary conferences to catch for web technology and now easier to engage with than ever before.

Links

Standard
Software

State of the Browser 2021

This is the first in-person conference I’ve been to since the pandemic and since it normally clashes with PyCon UK this is also the first State of the Browser that I’ve been too in a while.

As a high-level pitch for the conference it a chance to hear from standard makers and browser developers about their thoughts on the web, web standards and issues in web development.

The conference had an audience of probably a third of what I felt it had the last time I attended in person. There was not issue with distancing and you could add a stickers to your attendee batch to nix photography and to ask for people to keep their distance.

Usually the chance to socialise and network is a major part of the conference experience but once I was there I realised that I didn’t really want to spend the time required to get to know someone new while COVID is as prevalent as it is, not attend the generous post-conference drinks.

Which made me wonder why I was there at all. The answer, on reflection, is that being physically present meant that I was actually present for the talks as well. I’ve bought tickets for virtual events earlier in the year and I still haven’t watch the videos.

By physically turning up I did pay more attention and I did engage and learn more than I did virtually.

I found a few things about the conference frustrating though. Firstly a number of the speakers weren’t there and instead had recorded a talk so being at the conference ended up being a collective video watch without being able to control the video and skip the boring bits. Also there were no questions from the audience because that was being handled on Discord. Now most of my Discord is taken up with gaming because, y’know that’s what Discord pretty much is for the most part. So I wasn’t able to see that side of things because I didn’t have time to set up some kind of work account. But generally whether it was Slack or something else I kind of think having the questions on the conference chat meant that the talks were actually lectures and where the speakers weren’t that proficient with their delivery it made the talks more boring.

So at the end of the experience I have no idea as to whether my attendance was a good idea or not. I probably would have been distracted at home but at least I could have sorted out Discord and have watched the pre-recorded videos in a more comfortable environment (I certainly could of dodged the morning torrential rain).

But when there was a good in-person speaker it was great. Rachel Andrew was the standout for managing a review of the history of layout systems while also previewing the thinking of the standards groups. In particular drawing a fascinating line between the necessity of the contains CSS directive to the ability to be able to look forward to container queries. Stephanie Stimac shared similar insight into what the future may hold for the development of the Form elements and their backwards-compatible codification and customisation.

Alex Russell offered a rebuttal of the locked down mobile ecosystems from a capitalist perspective but failed to really offer remedies given that this overall is a capitalist failure.

In a totally different vibe Heydon Pickering did a talk about requiring people to switch off Javascript to read his blog. It was closer to standup and I did laugh out loud several times although trying to explain what made it funny and entertaining has proven highly difficult.

Rachel Andrew is one of the people behind Notist which a few people were using to share slide links. I hadn’t heard of it before and I can see it’s pretty handy compared to trawling Youtube trying to figure out if some talk you half remember has been posted there.

Overall I think it was worth the effort, I felt I got outside my bubble for a while and felt a bit more connected to the efforts that are still ongoing to safeguard and advance the web as a whole.

Standard
Programming

Sustainable web development

This article about delivering a sustainable small website is one of the most inspiring things I’ve read in weeks. It made me think about the true costs of the web revolution and made me think again about the environmental claims we make about moving paper processes online. Green energy and clean websites are entirely possible but will people pay the premium for it?

Finally like all truly great pieces it offered some practical insight into problems I have been grappling with in my work. The observation that small, fast loading server-rendered pages are the same as client-side SPAs but more energy efficient is a great insight.

However the best was the idea of simplifying the navigation into its own page. This one thing is a piece of genius that hundreds of man-years of UX research have failed to discover.

Thinking about the energy cost of our websites is also a great proxy for thinking about the cost basis for our infrastructure and where we want to put our processing effort.

Standard
Programming, Python

403 Forbidden errors with Flask and Zappa

One thing that tripped me up when creating applications with Zappa was an error I encountered with form posting that seems to have also caught out several other developers.

The tldr is that if you are getting a 403 Forbidden error but your application is working locally then you probably have a URL error due to the stage segment that Zappa adds to the URL of the deployed application. You need to make sure you are using url_for and not trying to write an absolute path.

The stage segment

Zappa’s url structure is surprisingly complicated because it allows you to have different versions of the code deployed under different aliases such as dev, staging and production.

When running locally your code doesn’t have the stage prefix so it is natural to use a bare path, something like flask.redirect(‘/’) for example.

If you’re using the standard form sequence of GET – POST – Redirect then everything works fine locally and remotely until the raw redirect occurs remotely and instead of getting a 404 error (which might tip you off to the real problem more quickly) you get a 403 forbidden because you are outside the deployed URL space.

If you bind a DNS name to a particular stage (e.g. app-dev.myapp.com) then the bare path will work again because the stage is hidden behind the CloudFront origin binding.

Always use url_for

The only safe way to handle URLs is for you to delegate all the path management and prefixing to Zappa. Fortunately Flask’s in-built url_for function, in conjunction with the Zappa wrapper can take care of all the grunt work for you. As long as all your urls (both in the template and the handlers) use url_for then the resulting URLs will work locally, on the API Gateway stages and if you bind a DNS name to the stage.

If this is already your development habit then great, this post is irrelevant to you but as I’ve mostly been using Heroku and App Engine for my hobby projects I’d found myself to be in the habit of writing the URLs as strings, as you do when you write the route bindings.

Then when the error occurred I was checking the URL against my code, seeing that they matched and then getting confused about the error because mentally I’d glossed over the stage.

Standard
Programming

Using Google App Engine with Pyenv

I recently started using PyEnv to control my Python installations and make it easier to try to move more of my code to Python 3.

Google App Engine though is unapologetically Python 2.7. Google wants people to move away from the platform in favour of Google Compute custom environments and therefore has little incentive to upgrade the App Engine SDK and environments to support Python 3.

When I set my default Python to be Python 3 with PyEnv I found that despite setting a local version of Python 2.7 my App Engine instance was failing to run with an execfile is not defined exception.

The App Engine Python scripts use #!/usr/bin/env python to invoke the interpreter and for some reason PyEnv doesn’t seem to override the global setting for this despite it being correct when you check it in your shell.

After a lot of frustration and googling for an answer I haven’t found anything elegant. Instead I found this Stack Overflow answer which helpful explained that you can use #!/usr/bin/env/python2 to invoke a specific language interpreter.

Manually changing the shebang line in appcfg.py and dev_appserver.py solved the problem for me and got me running locally.

Obviously this is a pain if I upgrade and I feel it might be better for Google to change the scripts since they don’t have a plan to move to Python3.

Standard