Blogging

Python x Rust meetup October 2024

This was the inaugural (hopefully) Rust/Python meetup crossover and to be honest it was heavy on the Python developers incorporating Rust into their code.

The talks were mostly recycled from other contexts (but no worse for having a bit of polish): David Seddon Crabs in Snakes!, David Hewitt How Python harnesses Rust, Samuel Colvin Pydantic v2 with Rust.

If pushed for time I would probably recommend David Hewitt’s one if you want to learn about how PyO3 (and foreign calls in Python generally) works and David Seddon’s if you just want to try and get a working Python-Rust interop.

During the Q&A Samuel made probably the key point of the evening for me that virtually anything you do in Rust is going to be faster than Python (and for good reasons, this isn’t a diss on Python). The call overhead is negligible and you’ll be using a system language. His case in point was string validation, even relatively simple validation is faster in Rust. David Hewitt’s example was Array processing versus Python’s dynamic Lists.

The disadvantages that there were mentioned about Rust was its complexity, which is simplified when the context is a function call and it being hard to try out different ideas and reorganise code, the strict compiler means that you need to move from one complete implementation to another.

If interop becomes increasing seamless (and it sounds like the biggest problem currently is having to build binaries) then there will be no reason not to drop from Python into Rust. There also won’t be a strong reason for not starting your programming in a high-level abstraction and dropping down a level when you need to.

Standard
Blogging

September 2024 month notes

Unfortunately I’ve been ill with acute bronchitis this month which meant not being able to attend a few events I’d been looking forward to and not sleeping very well which meant not much time to do anything that wasn’t essentially.

I have some posts I’m working on around class-based CSS systems and working with datamapper solutions that allow you to work with SQL directly compared to ORM abstractions but they aren’t ready as yet.

I did attend the CTO Unconference which was another good event but with less actionable ideas than previous sessions. There were a few conversations about good development processes and what core things a technical solution should provide. We didn’t really get into costs and focus but my view was that people generally didn’t think in terms of cost-benefit and that there is a lot of zero-interest-rate legacy that we are now going to have to deal with in a very different financial era. Simplicity and a minimum of moving parts seems important.

I’m a big fan of developing in the open and I think the organisations I’ve worked for that adopted this practice were the best places I’ve worked at for communication and design. Ross Ferguson (a former colleague) is compiling a list of public organisations that use open roadmaps which are a really interesting idea (if probably a bit scary for organisations that don’t have disclosure or transparency requirements)

Gitlab does open support tickets (Background jobs failing, Background job results not being available) which was something brand new to me. Again it seems quite scary but on seeing it working it gave me a tremendous amount of insight into the problems and what the Gitlab folks were trying to do about it. Something that no amount of strongly worded emails to support addresses or account managers has ever gotten me.

Standard
Blogging

State of the Browser 2023

The State of the Browser conference is an annual staple for me (although it clashes with the excellent Pycon UK) and this year’s edition was another good day out. The logistics of the conference are great with live captioning on the talks, a live stream and the videos going online quickly after the event. If you attend in person then the new venue at the Barbican is comfy and well-organised, the only glitch being that the breakfast rolls (particular the vegetarian ones) ran out very quickly.

The keynote was a bit poor and rambly but did make a good point that of the top 100 websites none has completely valid HTML which seems crazy but also points to how hard it can be to create a complex website.

Amy Hupe‘s It all means nothing in the end was the only talk that was about personal experience rather than technology and I was grateful for that because at some conferences it can be a third of the content these days. The talk is absolutely fine, I don’t think it gets the definition of burnout quite right because the workplace is an integral part of the definition and therefore for a self-employed contractor I think you’re not really talking about burnout but about the expectations that you put on yourself. I was also interested that maybe people are only being exposed to OKR-style stretch goals at work now and maybe incremental ways of working, getting things done techniques and tiny habits are not as well known anymore.

Ian Lloyd‘s talk on UI accessibility horror stories was funny and it was very thought-provoking about how bad the theatre and cinema seat pickers are in particular that the accessible seating is not identified as such to assistive technology. However the second half of the talk lacked a bit of clarity about how to breakdown more complex UIs, in particular I was curious to know about how detail can be exposed progressively as needed.

Killian Valkhof’s talk on how common requirements can be met with pure standards-based techniques was the best technical talk of the day for me but it was absolutely blighted by a technical issue with the monitor connection. I hope he blogs about some the ideas he shared on the day because they are a lot simpler than a lot of approaches I’ve seen. I think the major takeaway from his talk for me was that our knowledge gets fossiled at the point we learn how to do something. It is harder to relearn something with new techniques than to learn a new thing altogether.

Diego González was the only representative of a browser maker this year, this input if vital to the conference so it would be great to have something from Mozilla in future years. His talk about PWAs involved an extended pastiche of Blue Planet which was amusing but I didn’t take that much away from it on a personal level. It is interesting to know that there is some self-reflection going on about what constitutes a progressive web apps and what their right relationship with native apps is. As compere Dave Letorey said during the conference “the browser is the everything app” if you’re not attempting to create a platform capitialist business then how much do apps matter to you compared to access to services and information?

The talk on caching by Harry Roberts was a great overview of cache headers which are the kind of thing you think you understand but I appreciated the clarity that you can just delete everything except Cache-Control and ETag and then map the behaviour into Cache Control directives with ETags as a bonus if relevant.

The final talk was a technical demo that used the speech to text API that is built-in to browsers from Apple and Google but isn’t really a web standard. The talk highlighted the huge burden that is put on Mozilla to make this something that can geuninely be open and used across implementations. To be honest I’ve seen talks like this before and they are fun but I’m not sure that this was the right forum if there wasn’t going to be a discussion about the compromise that is required to share your voice data with tech giants.

Chris Ferdinandi provided pre-recorded inserts between talks which were very professionally done (as you might expect from someone who does it for a living). He ignited a new interest in Web Components in me. I had previously been a bit sceptical but now they are fully supported it feels silly to use even a microframework if there is a standard available.

The attendees were very friendly and the environment as attractive as ever and the tickets are really a bargain although there was an appeal from the organisers for people to buy the tickets early to avoid them being personally liable for the deposit. Another Saturday well-spent.

Finally a shout out to the makers of Invidious for making it easier to review the videos from the day in a logical interview than actual Youtube. I’m sure it can’t be long before it gets shutdown somehow.

Links

Previous reports

Standard
Blogging

RSS Readers for the mobile web

After every social media convulsion there is always a view that we’re heading back to blogs again. Regardless of whether this is true or not there is always an uptick in posting and blogs are definitely better for any kind of long form content compared to a 32 post “thread” on any kind of microblogging social platform. So I’ve been revising my line up of RSS readers (like email I use a few) and I wanted to post my notes on what I’ve tried and what I’ve ended up using.

My first key point of frustration is viewing content on a phone browser. My primary reader (which I migrated to from Google Reader) is Newsblur but the design of the site is not responsive and is large screen focused. My second issue is specifically around Blogger sites, while these do have a mobile view most of the themes for Blogger feel unreadable and harsh on smaller screens. Not to mention the cookie banner that is always floating around.

I have been using Feedbin whose main feature is that it can consolidate content from Twitter, RSS and email newsletters into a single web interface. It does deliver this promise but while its small screen experience and touch interface has been considered, the resulting UI is quite fiddly with a side-swipe scheme for drilling in and out of content and I often need to switch out of its default rendering mode to get something that is easy to read. I’m still using Feedbin to follow news sources on Twitter but have mostly given up on RSS there except indirectly through topic subscriptions.

I want to give an honourable mention here to Bubo RSS. This is essentially a static site builder that reads your subscriptions and builds a set of very lightweight pages that list out all the recent posts and uses the visited link CSS property to indicate the unread items. In the end this didn’t really solve my reading issues as you just link through to the original site rather than getting cleaned up small screen friendly view. However its idea of building a mini-site from your RSS feed and then publishing a static site would solve a lot of my problems. I was almost tempted to see if I could add a pull of the content and a Readability parse but I sense the size of the rabbit hole I was going into.

Another great solution I found was Nom which is a terminal RSS reader written in Go. You put your subscriptions into a config file and then read the content via the terminal. If I had any feedback for Nom it would be that the screen line length is not adjustable and the default feels a bit short. The pure text experience was the best reading experience for the Blogger subscriptions I have but ultimately I wanted something that I could read on a mobile phone web browser.

In the end the thing that has been working for me was Miniflux. You can self-host this but the hosted option seemed cheaper to me than the cost of the required hosting. I had only one issue with Miniflux’s reading mode out of the box which was to do with margins on small screens, I thought I might have to try and get a PR organised but helpfully you can save a custom CSS snippet in the settings and with a few lines of customisation I was entirely happy with the result. This is now what I’m using to read RSS-based content on my phone.

Standard
Blogging, Python

PyCon UK 2022

This was the first in-person PyCon since the start of the pandemic. It had slightly changed format as well, now being mostly single-track (plus workshops) and not having a teacher/youth day. Overall I found the changes welcome. I’m generally a big fan of single track conferences because they simplify the choices and help concentrate the conversation amongst the attendees.

A lot of talks were by developer advocates but some of the most interesting talks came from the core language maintainers talking about the challenges in balancing the needs of different parts of the community while the least interesting were from the company sponsors who generally didn’t seem to have refined their content for a general audience.Typescript

A simple lightning talk about the need to sanitise the input of the builtin int function was illuminating about the challenges of reconciling web needs versus data science. However the general point about whether unlimited number functions are necessary or not was interesting. Similarly the challenge of when to adopt new syntax and keywords in the case of the addition of Exception Groups to the language.

There were a few talks about how the process of maintaining a language and community actually works. How conferences get listed on the Python website, how performance benchmarks are built and used and the desire to have a single place to centre conversation about the language.

There were talks about how to interface to various technologies with Python (Kafka, Telegram) and the inevitable talk about how to improve the performance of your data science code by using NumPy. There were also quite a few talks about Hypothesis; which probably still isn’t used as much as it should be in the community. The library now includes a test generator that can examine code and suggest a test suite for it which is quite a cool feature and certainly would be a boon for those who dislike test-first approaches (whether it has the same quality benefits is another question).

The other talk that had a big impact on my was the introduction to using PyScript in static websites. Python is going to start targeting WebAssembly as a platform which will help standardise the various projects trying to bring a fully functional interpreter to the browser and provide a place to pool efforts on improvements.

PyScript aims to allow Python to also script DOM elements and be an alternative scripting language in the browser. That’s fun but not massively compelling in my view (having done many compile to Javascript languages in the past I think it’s easier to just do Javascript (with an honourable exception for Typescript). Being able to run existing code and models in the browser without change maximises the value and potential of your existing Python codebases.

Cardiff remains a lovely venue for the conference and I think it is worth taking time out from the tracks to enjoy a bit of the city and the nearby Bute Park.

Links

  • Pyjamas, a 24 hour online conference
  • Pyrogram, a Python library for interacting with Telegram
  • HPy, an updated API for writing C extensions in Python
  • Discuss Python, the general Python community forum
  • Trustme, a testing library that allows local certificates as testing fixtures
  • PyScript, Python in the browser via Web Assembly
  • PyPerformance, benchmarks for testing Python implementations
Standard
Blogging, Work

Trivialising stress with “Wellness at Work”

Burnout is a complex topic but employer responses to it are even more complex. Some definitions of burnout make it a direct occupational health hazard and therefore something that all employers need to consider and manage.

But the way that many companies are responding is facile and, as many other people have already pointed out, patronising to employees and minimising the employer’s own responsibilities. Occupational stress is not something that is resolved by reminders of “mindfulness” and the offer or an meditation app and an employee helpline.

Stress at work is probably inevitable for most workers employed by organisations. The friction and pressures of work continually generate stress which needs to be managed through conventional tools like support and leave. An employee does need to be aware of the indicators and behaviours associated with their own stress but this is probably something that requires some individual coaching rather than an app and a video lesson.

By offering inadequate or lackluster support employers seem to put themselves in the worst possible position, acknowledging their responsibility for the occupational nature of the problem and yet not offering a meaningful solution to it. Ultimately some reckoning seems inevitable in this situation.

A deeper review of the origins of stress within the organisation is needed. Things like client interactions, challenging targets, arbitrary deadlines, interpersonal issues may all be deemed a necessary part of an organisation’s management but acknowledging the consequences of these sources of stress and figuring out how the consequences can be managed and ideally neutralised through more positive activities is vital management work.

Commercialised wellness is an almost meaningless marketing concept, mixing it with the realities of workplace stress seems a recipe for disaster rather than the cheap, effortless fix some employers seem to think it is.

Standard
Blogging, Programming

Clojure Exchange 2016

At one point during this year's Clojure Exchange I was reflecting on the numerous problems and setbacks there had been in organising the 2016 exchange with Bruce Durling and he simply replied: "Yeah it was a 2016 type of conference". So that's all I really want to say about the behind the scenes difficulties, despite the struggles I think it was a decent conference.

Personal highlights

James Reeves's talk on asynchronous Ring was an excellent update on how Ring is being adapted to enable asynchronous handlers now and non-blocking handlers in the future. I didn't know that there isn't an equivalent of the Servlet spec for Java NIO-based web frameworks.

The Klipse talk is both short and hilarious with a nicely structured double-act to illustrate the value of being able to evaluate code dynamically on a static page.

David Humphrey's talk, Log all the things was pretty comprehensive on the subject of logging from Clojure applications. It was one of those talks where you felt "well that's been sorted then".

Both Kris's keynote and Christian's Immutable back to front talked not just about the value of Clojure but how you can apply the principles of Clojure's design all across your solution.

One of the most interesting talks was a visualisation of prisoner's dilemma strategies in the browser. It was visual, experimental and informative.

Henry Garner's data science on Clojure talk was interesting again with some nice dynamic distributions and discussions of multi-arm bandit dynamic analysis. Sometimes I feel lots of the data science stuff is too esoteric with too little tangible output. This talk felt a little more relatable in terms of making dynamic variant testing less painful.

Disappointments

Not everything sings on the day. Daan van Berkel's talk on Rubik's Cubes suffered a technical failure that meant his presentation was not dynamically evaluating and therefore became very hard to follow. We should have tried to switch talks around or take a break and try and fix it.

The AV was a general rumbling problem with a few speakers having to have a mic switch in the middle of their talks.

Hans Hubner's talk on persistence was interesting but too quick and too subtle.

We should have had the two Spec talks closer together and earlier in the day. The things that people are doing with it are non-trivial and it is still a relatively new thing.

clojure.spec

Spec is kind of interesting generally for the community. It has become very popular, very quickly and it is being used for all kinds of things.

One theme that came up in the conference was the idea that people wanted to share their spec definitions across the codebase. This seems a bad idea and a classic example of overreach, if someone said they defined all their domain classes in a single Java jar and shared it all across the company then you'd probably thing that is a bad idea. It's not better here because it is Clojure.

The use of Spec was also kind of interesting from a community point of view as the heaviest users of Clojure seemed to be doing the most with it. The bigger the team and the codebase the quicker people have been to adopt Spec and in some cases seem to switch from using Schema to Spec.

On the other hand the people using Clojure for data processing, web programming and things like Clojurescript have not really adopted Spec, probably because it simply doesn't add a lot of benefit for them.

So for the first time in a while we have something that requires some introduction for those new and unfamiliar with it but is being used in really esoteric ways by those making the most use of it. There is a quite a big gap between the two parts of the community.

The corridor track

Out of the UK conferences I went to Clojure Exchange felt like it had the best social pooling of knowledge outside of Scale Summit. Maybe it was because I knew more people here but the talks also had all kinds of interesting little tips. For example during Christian's talk he mentioned that S3 and Cloudfront make for one of the most reliable web API deployment platforms you can choose to use. I ended up making a huge list of links of reminders and things to follow up on. I've also included links to lots of the Github repos that were referenced during the talks.

Next year

And so with a certain inevitability we are looking to the next Clojure Exchange. We're going to have a slightly bigger program committee which should make things easier.

The other thing that we didn't really do that well this year was to try and have some talks transfer from the community talk tracks to the event. In 2017 we'll hopefully be more organised around the community and also have a series of talks that are tied in to the conference itself. If you're interested in being involved in either the organising or the talks you can get involved via London Clojurians.

See you there!

Standard
Blogging, Programming, Web Applications

An overview of Javascript reactive frameworks

This post is only meant to be a snapshot of the current state of the various DOM virtualising webframeworks that are around. I’m partly publishing it to try and discover more that I may not be aware of.

Many of these frameworks trace an ancestry back to Om and React. However each one tries to deal with perceived problems with the original frameworks. The most common being that React is too heavy and opinionated while not providing a consistent data model for components. Om on the other hand is in Clojurescript and therefore represents too much to learn in terms of a new language and build process.

Libraries

Most of the libraries build on a few common building blocks that I’m not going to elaborate on here. Virtualdom was an early attempt to separate the core idea of React from the rest of the library code. Virtualdom is only concerned with creating, manipulating and stringifying DOM structures in-memory. Browser DOM APIs involving linking to the actual rendered document so managing virtual DOM is more efficient and simpler because you’re not interacting with these underlying libraries.

ImmutableJS provides a Javascript-idiom interpretation of the Clojure data structures that Om uses (and which are available as the standalone library Mori).

Omniscient

The first interesting framework to discuss is Omniscient, which as its name suggests is heavily influenced by Om but is written in Javascript and therefore does not require you to learn Clojure to use the same techniques that Om uses. Omniscient is built on top of React and ImmutableJS and uses its own library Immstruct to add reference cursors to ImmutableJS structures. Reference cursors allow a component to observe and change sections of a data structure without having to manipulate the whole thing. So for example a component can be given a single sub-key in an object that represents its state and it cannot access or change anything that is not under that key. The code can also be simplified to behave as if the sub-key was actually just the whole data object.

Omniscient doesn’t suggest an alternative to Om’s CSP, instead providing a mechanism for passing event flow functions down the component tree. You’re free to choose your own event libraries. It also means that you’re free to make your own mistakes here as no guidance is really given as to how to structure your event scheme appropriately.

Omniscient is one of the earliest frameworks to re-implement Om and therefore has one of the better sets of documentation on its Github pages. That said there’s not a lot of documentation and the framework does not have a massive community. The situation is worse in most of the other frameworks though so this might tip you over in favour of Omniscient.

Ractive

This is a bit of a Guardian shout out as the primary developer Rich Harris is a Guardian interactive developer.

Ractive (Github) is a little be different from the other frameworks as you can essentially think of it as Mustache templates backed by Observables. You declare a data-binding and write templates in normal Mustache syntax but behind the scenes Ractive is driven by changes in the data and then writes new section of DOM in-memory according to what has changed rather than DOM diff’ing.

Also Ractive sticks with two-way databinding rather than unidirectional data flow so failures in synchronisation or rendering can be problematic.

If what you want to do is render content over a Javascript data model then there is a lot in Ractive that is very compelling. It uses templates with a standard syntax that is well understood and is a soup and nuts framework that sticks to core Javascript syntax and features. However if you want to use your own event or data model you are out of luck.

Mercury

Mercury on the other hand prides itself on modularity. A microframework it attempts to create a glue layer that allows other libraries to interact in a sensible and consistent way. The default components are Virtualdom and its own observer pattern to wrap state.

Mercury’s biggest problem right now is its lack of documentation. There is an expectation that you are going to read the source code to understand what the framework is doing and how to interact with the API. I frankly think this is unrealistic. The project doesn’t currently supply the incentive to do that. Unless you have a very particular desire to avoid any framework lock-in or you want to use a very specific combination of libraries that is not supported elsewhere its hard to understand why you would invest your effort here rather than in frameworks that offer more support.

Cycle

Cycle is similarly experimental, its biggest claim is that it is truly reactive and that the rendered page is purely the result of change in state. The introduction is couched in computer science theory but it would seem that at its heart Cycle wraps RxJS and Virtualdom in a glue layer that has the programmer writing the transform sequence between the event and the DOM structure.

I think it is a positive feature that Cycle re-uses a popular library to manage its state-transitions rather than implementing yet another custom version of the Observable pattern. It also makes the framework easier to get started with if you are familiar with the Rx.

Using established libraries also makes the lack of documentation more acceptable as the Cycle readme only needs to explain how the glue works in the framework.

As something built on reactivity you have to get used to dealing with intermediate state which can be bit difficult for the beginner.

Essentially any event where the user would expect feedback means you need write the conditional structure in the output. So if the user types a character in an input box then you need to write the value of the input box to be the characters the user has typed so far. Most frameworks work at a higher level of abstraction or rather they map closer to the DOM APIs, so getting a working application means grokking the way the dataflow works.

If you’re looking for purity (and a resulting simplicity in implementation) but not to have to learn a bespoke API Cycle is nicely positioned.

WebRx

WebRx is similarly built on top of RxJS Observables but is a much fuller-fat framework that is much more a spiritual successor to Knockout than owing much to the influence Om or React.

Rather like React WebRx doesn’t really provide generalised event handling but instead has special sauce bindings for DOM events and a MessageBus system built over Rx.

It is also written in Typescript and generally looks to play well within the Microsoft ecosystem. It’s interesting to me as an example of how different a language has to be before its regarded as a barrier. Clearly the use of Typescript means there are people who will refuse to use the framework regardless of whether it works for their use case. Other people are going to be attracted exactly because it uses Typescript.

Deku

Language choices are also interesting in Deku which is another attempt to re-implement React in a superficial way.

Deku makes use of ES6 and 7 features and doesn’t aim to support a broad range of browsers (unlike say Ractive). Again that is going to rule it out for some people but this is a more interesting as now we are within dialects of the same core language. Language choice for implementing frameworks is not straightforward. What are you looking for? Conciseness? Editor support?

Deku aims to take the dom diffing approach but avoid getting caught in React’s framework and approach. In particular components are defined just as Javascript objects rather that classes and instances. Something I think makes it more elegant that normal React Components.

It does however still use JSX which is quite interesting as the framework claims to be taking a functional approach but actually uses a DSL for all its DOM construction.

The lifecycle hooks are slightly different with more hooks for different stages of the process and Deku uses some interesting function passing to send changed data down the tree to components.

Deku doesn’t take much influence from Om though. It doesn’t have sophisticated event handling and uses mutable data with generous access and callbacks on data write to do re-renders. This means bugs and state issues are no less likely to happen than with any other framework. It does adopt the single atom idea with a single tree representing the app and the app renderer being bound to the body element.

As such if you like the idea of React but don’t want to bound into its concept of how a Component should be defined but do like JSX and trust the implementors to create a better dom diff than Facebook or Virtualdom, this is the project for you.

Conclusion

I’ve only chosen a handful of frameworks to look at here, mostly based on the ones I know, I’m expecting people to point out more in the comments. I also haven’t used all of these frameworks. Road-testing all of them would be a bigger task than just trying to describe the design choices they’ve made.

The most common pattern is to try and improve the rendering time versus React by using different virtual dom difference algorithms. Usually this is combined with Observed variables that provide a Reactive component that allows changes in the data model to be conveyed to the DOM model with no coding required.

Few of the frameworks engage with the functional reactive programming paradigm by building abstract event streams or indeed any abstraction over discrete events.

The idea that the app should be a single data structure that represents the whole page seems to be gaining significant traction with several of the frameworks recommending this as an approach.

The explosion of frameworks resulting from the release of React is, I think, a positive thing. Initially it seems really daunting that you have all these choices but when you look at the real level of difference between them you can see that they are actually quite tightly coupled around a few common and core ideas and that mostly they express differences about the concerns that a framework should have which feeds into the wider conversation about micro or comprehensive frameworks.

Standard
Blogging

Stemming the spam

One of the features I particularly value on WordPress has been the ability to have fairly open comment threads on my blog posts. In the past Askimet used to keep these spam free and people were able to submit comments fairly freely.

Over the course of this year though, comment spam has been getting vaguely more sophisticated. The most common spam has been backlink SEO spam that consists of reasonable emails and pieces of genuine comments mixed with product keywords and generic praise.

This kind of spam seems to be massively problematic to detect. Large numbers of comments have been made to historic posts that presumably have ended up ranking well in Google.

Moderating that stuff has been a chore and if I am away from the internet for any period of time the comment were on the site for a while and this seemed to be encouraging more. I presume the bots are smart enough to detect when their comemnts have not been deleted for more than eight to twelve hours and then they pile in.

The only reasonable way forward seems to me to be switch on someone kind of pre-moderation. I’ve gone for the mildest level of control that seems to work, namely that the first post by a user will now be moderated but once someone has a valid post they will be able to post freely.

This seems to strike the right balance between allowing people to tell me I know nothing about Clojure, Scala or Javascript and avoiding people having to sign-in to post comments.

As for the creators of these ingenious bots, a decidedly slow handclap; well-played.

Standard