Programming, Software, Work

Defining the idea of “software engineering”

I have been reading Dave Farley’s Modern Software Engineering. Overall it’s a great read and thoroughly recommended (I’m still reading through it but I’ve read enough to know it is really interesting and a well-considered approach to common problems in development).

One of the challenges Dave tackles is to try and provide a definition of what software engineering actually is. This is actually a pretty profound challenge in my view. I’ve often felt that developers have usurped the title of engineer to provide a patina of respectability to their hacky habits. Even in Dave’s telling of the origin of the term it was used to try and provide parity of esteem with hardware engineers (by no lesser figure than Margaret Hamilton).

In large organisations where they have actual engineers it is often important to avoid confusion between what Dave categorises as Design and Production engineering. Software engineering sits in the world of design engineering. Software is malleable and easy to change unlike a supply chain or a partially completed bridge. Where the end result of the engineering process is an expensive material object Dave points that it is common to spend a lot of time modelling the end result and refining the delivery process for the material output as a result of the predictions of the model. For software to some extent our model is the product and we can often iterate and refine it at very low cost.

Dave proposes the following definition of engineering:

Engineering is the application of an empirical, scientific approach to finding efficient, economical solutions to practical problems.

Dave Farley, Modern Software Engineering

This definition is one I can live with and marries my experience of creating software to the wider principles of engineering. It also bridges across the two realms of engineering, leaving the differences to practices rather than principles.

It is grounded in practicality rather than aloof theories and it emphasises that capacities drive effective solutions as much as needs. This definition is a huge step forward in being able to build consensus around the purpose of a software engineer.

Standard
Programming

State of the Browser 2022

I’ve attended a few of these conferences and have always found them helpful. This year it had relocated to the Barbican Centre with the food and drink area overlooking the beautiful Conservatory there, great choice as a venue.

The conference was a hybrid in-person/online event that I think could serve as a model for other conferences that seem to only be focusing on their return to in-person. Due to other commitments I wasn’t able to be at the venue all day and so at lunchtime I headed home and picked up a few of the rest of the talks on the livestream. It was great to have the flexibility and made the whole conference more accessible.

Talks-wise it was interesting as ever and a little bit less inward looking or niche interest that it has been in the past. There were the usual mix of upcoming standards and challenges in implementing them, how to apply techniques to the current broad mainstream of browsers and a little bit of evangelism for playfulness and environment impact.

One of my key takeaways was on this last point; using an image CDN that can do automatic content negotiation to use an efficient modern image standard has a huge carbon saving. It feels a bit crazy that so many companies are still serving fixed sizes and formats off things like Cloudfront and S3.

Bruce Lawson kicked off the event with a good historical perspective talk on the history of standards (and the struggle to create and maintain them) and brought the issues of standardisation through the search for technical solutions to the world of regulation and better digital policy. Engaging with law makers is a more realistic way to improve the online world that the search of technical solutions to social problems.

More practically we can hope that Apple will be compelled as a digital gatekeeper to allow competition on browser implementations on its platform and maybe even fund its Safari team properly to have better compatibility with the general web standards on iOS. I felt it was nice for a recognition that government organisations can be engaged and willing to listen and that progress can be made be working together rather than outside of regular power structures.

Probably the best talk I heard was “Be the browser’s mentor not it’s micromanager” by Andy Bell this talk neatly encompassed two major ideas: the first was the way that layout systems in CSS have advanced to the point where you are describing structure and allowing the layout manager to actually decide the rendering and secondly on how digital design approaches have managed to fall between the abstractions of the grid system and the precise layout of magazine style layout.

By leaning on the layout engines the amount of CSS we have to write is much more minimal than the micromanaging fussiness typical to component design systems. It is also more powerful and expressive, avoiding the overly complex muddle that is often associated with component style systems but also not going too far down the class frenzy of utility class systems.

Sophie Koonin taught me how to use the prefers-reduced-motion preference via the medium of late 90s website chaos. A good example of the mixture of fun and practical content.

I also enjoyed Alistair Shepherd‘s talk which had a few technical bits and pieces but managed to bridge the themes of the conference by wanting to create a personal website that first and foremost reflected his personal interests and then used the tech to deliver the vision he had for himself. Although the idea to have websites that vary according to the time of date is quite an interesting idea.

I didn’t catch the last few talks so I’m hoping to be able to watch them when they come to YouTube (or maybe some federated alternative!).

Overall still one of the necessary conferences to catch for web technology and now easier to engage with than ever before.

Links

Standard
Blogging, Python

PyCon UK 2022

This was the first in-person PyCon since the start of the pandemic. It had slightly changed format as well, now being mostly single-track (plus workshops) and not having a teacher/youth day. Overall I found the changes welcome. I’m generally a big fan of single track conferences because they simplify the choices and help concentrate the conversation amongst the attendees.

A lot of talks were by developer advocates but some of the most interesting talks came from the core language maintainers talking about the challenges in balancing the needs of different parts of the community while the least interesting were from the company sponsors who generally didn’t seem to have refined their content for a general audience.Typescript

A simple lightning talk about the need to sanitise the input of the builtin int function was illuminating about the challenges of reconciling web needs versus data science. However the general point about whether unlimited number functions are necessary or not was interesting. Similarly the challenge of when to adopt new syntax and keywords in the case of the addition of Exception Groups to the language.

There were a few talks about how the process of maintaining a language and community actually works. How conferences get listed on the Python website, how performance benchmarks are built and used and the desire to have a single place to centre conversation about the language.

There were talks about how to interface to various technologies with Python (Kafka, Telegram) and the inevitable talk about how to improve the performance of your data science code by using NumPy. There were also quite a few talks about Hypothesis; which probably still isn’t used as much as it should be in the community. The library now includes a test generator that can examine code and suggest a test suite for it which is quite a cool feature and certainly would be a boon for those who dislike test-first approaches (whether it has the same quality benefits is another question).

The other talk that had a big impact on my was the introduction to using PyScript in static websites. Python is going to start targeting WebAssembly as a platform which will help standardise the various projects trying to bring a fully functional interpreter to the browser and provide a place to pool efforts on improvements.

PyScript aims to allow Python to also script DOM elements and be an alternative scripting language in the browser. That’s fun but not massively compelling in my view (having done many compile to Javascript languages in the past I think it’s easier to just do Javascript (with an honourable exception for Typescript). Being able to run existing code and models in the browser without change maximises the value and potential of your existing Python codebases.

Cardiff remains a lovely venue for the conference and I think it is worth taking time out from the tracks to enjoy a bit of the city and the nearby Bute Park.

Links

  • Pyjamas, a 24 hour online conference
  • Pyrogram, a Python library for interacting with Telegram
  • HPy, an updated API for writing C extensions in Python
  • Discuss Python, the general Python community forum
  • Trustme, a testing library that allows local certificates as testing fixtures
  • PyScript, Python in the browser via Web Assembly
  • PyPerformance, benchmarks for testing Python implementations
Standard
Blogging, Work

Trivialising stress with “Wellness at Work”

Burnout is a complex topic but employer responses to it are even more complex. Some definitions of burnout make it a direct occupational health hazard and therefore something that all employers need to consider and manage.

But the way that many companies are responding is facile and, as many other people have already pointed out, patronising to employees and minimising the employer’s own responsibilities. Occupational stress is not something that is resolved by reminders of “mindfulness” and the offer or an meditation app and an employee helpline.

Stress at work is probably inevitable for most workers employed by organisations. The friction and pressures of work continually generate stress which needs to be managed through conventional tools like support and leave. An employee does need to be aware of the indicators and behaviours associated with their own stress but this is probably something that requires some individual coaching rather than an app and a video lesson.

By offering inadequate or lackluster support employers seem to put themselves in the worst possible position, acknowledging their responsibility for the occupational nature of the problem and yet not offering a meaningful solution to it. Ultimately some reckoning seems inevitable in this situation.

A deeper review of the origins of stress within the organisation is needed. Things like client interactions, challenging targets, arbitrary deadlines, interpersonal issues may all be deemed a necessary part of an organisation’s management but acknowledging the consequences of these sources of stress and figuring out how the consequences can be managed and ideally neutralised through more positive activities is vital management work.

Commercialised wellness is an almost meaningless marketing concept, mixing it with the realities of workplace stress seems a recipe for disaster rather than the cheap, effortless fix some employers seem to think it is.

Standard
Programming

Keeping the batteries included

Python is well-known for being a “batteries included” language which means it comes with a rich variety of modules that work right out of the box. However this recent post about the regular expression library shows the problems that can occur from shipping such a wide variety of code as part of the language core. Maintaining a wide-ranging codebase is challenging as is keeping it aligned to a language’s release cadence.

The problems suggest that really the language should focus just on the core tools and syntax of the language with everything else being on a different release cycle. However by itself that core isn’t too useful for pragmatic coding purposes. Curation of code and having a sensible selection of libraries is a challenge. Some languages like Clojure and Elm have a controlled ecosystem of libraries that are adjacent to a small language core. However here the difference between curation and gatekeeping is fine and it feels like only languages with a large community can do it effectively.

Perhaps the answer is to have a basic implementation of core functionality in the core but to use the language documentation to suggest alternative libraries. This moves the problem to the documentation team but hopefully this is a simpler arena to both maintain and curate.

Standard
Software

State of the Browser 2021

This is the first in-person conference I’ve been to since the pandemic and since it normally clashes with PyCon UK this is also the first State of the Browser that I’ve been too in a while.

As a high-level pitch for the conference it a chance to hear from standard makers and browser developers about their thoughts on the web, web standards and issues in web development.

The conference had an audience of probably a third of what I felt it had the last time I attended in person. There was not issue with distancing and you could add a stickers to your attendee batch to nix photography and to ask for people to keep their distance.

Usually the chance to socialise and network is a major part of the conference experience but once I was there I realised that I didn’t really want to spend the time required to get to know someone new while COVID is as prevalent as it is, not attend the generous post-conference drinks.

Which made me wonder why I was there at all. The answer, on reflection, is that being physically present meant that I was actually present for the talks as well. I’ve bought tickets for virtual events earlier in the year and I still haven’t watch the videos.

By physically turning up I did pay more attention and I did engage and learn more than I did virtually.

I found a few things about the conference frustrating though. Firstly a number of the speakers weren’t there and instead had recorded a talk so being at the conference ended up being a collective video watch without being able to control the video and skip the boring bits. Also there were no questions from the audience because that was being handled on Discord. Now most of my Discord is taken up with gaming because, y’know that’s what Discord pretty much is for the most part. So I wasn’t able to see that side of things because I didn’t have time to set up some kind of work account. But generally whether it was Slack or something else I kind of think having the questions on the conference chat meant that the talks were actually lectures and where the speakers weren’t that proficient with their delivery it made the talks more boring.

So at the end of the experience I have no idea as to whether my attendance was a good idea or not. I probably would have been distracted at home but at least I could have sorted out Discord and have watched the pre-recorded videos in a more comfortable environment (I certainly could of dodged the morning torrential rain).

But when there was a good in-person speaker it was great. Rachel Andrew was the standout for managing a review of the history of layout systems while also previewing the thinking of the standards groups. In particular drawing a fascinating line between the necessity of the contains CSS directive to the ability to be able to look forward to container queries. Stephanie Stimac shared similar insight into what the future may hold for the development of the Form elements and their backwards-compatible codification and customisation.

Alex Russell offered a rebuttal of the locked down mobile ecosystems from a capitalist perspective but failed to really offer remedies given that this overall is a capitalist failure.

In a totally different vibe Heydon Pickering did a talk about requiring people to switch off Javascript to read his blog. It was closer to standup and I did laugh out loud several times although trying to explain what made it funny and entertaining has proven highly difficult.

Rachel Andrew is one of the people behind Notist which a few people were using to share slide links. I hadn’t heard of it before and I can see it’s pretty handy compared to trawling Youtube trying to figure out if some talk you half remember has been posted there.

Overall I think it was worth the effort, I felt I got outside my bubble for a while and felt a bit more connected to the efforts that are still ongoing to safeguard and advance the web as a whole.

Standard
Web Applications

Email services in 2021

I read this article about switching away from GMail and it struck a bit of a chord in terms of my own attempts to find a satisfactory replacement.

At the moment I feel I’m using all the services and at some point I should actually pick one or two that actually meet my need.

I have ProtonMail and Tutanota accounts for security. In truth I’ve ended up using neither (unless I see someone else using a ProtonMail address).

Day to day I’m probably still using GMail for most things and Fastmail for things where I don’t want to embarrass myself with my GMail address which is based on a gaming handle. Therefore over time my Fastmail address has become my address for financial things, communication with tradespeople and professionals and the odd real-world email invoice and so on.

It may sound strange but the biggest reason I don’t use Tutanota more is that it is hard to communicate verbally to other people. Fastmail still needs to the first word to be spelled out but people expect the XMail.com format and seem to have a lot less trouble with it.

I was on the verge of unsubscribing from Hey when it had it’s massive wobble over handling political issues at work (or alternatively white privilege, whichever way you see it). Rightly or wrongly I’ve kept on using and paying for it.

The strange niche that I’ve found for it is communicating with my extended family and occasionally a bit of business communication. The mail handling features just seems to work really well in terms of not wanting to respond immediately but wanting something better than “star” or “pin”.

When I started with Hey I was very excited about the “Feed” feature for managing email newsletters but after a while I’ve found myself not using it very much and instead I’ve started using Feedbin for these instead.

The Hey Paper Trail function is also good but when it comes to things like online orders I find the delivery updates easier to handle in GMail.

However exactly like the author of the article Fastmail is the most complete replacement for GMail having a similar functionality set (including a calendar) and while Hey might be better for having a well-managed near-zero mailbox, Fastmail is better for the pragmatic keep it all and search it when you need it approach to email.

Standard
Programming

PR Reviews: not the messiah, not a naughty boy

On Tech Twitter there seems to be a lot of chat recently about how the Github Pull Request (PR) review process is “broken” or underserves internal teams (an example) that have strong collaboration practices such as pairing or mob coding.

Another objection raised is the idea of internal gatekeeping within development groups, I’m not sure I fully follow the argument but I think it runs along the lines that the PR review process allows powerful, influential members of the group to enforce their views over the others.

This is definitely a problem but frankly writing AST tools linked to the “merge to master” checks is probably a more controlling tool than spending your time policing PRs.

With open source projects often all the action happens at the pull request because the contributors often have no other interaction. Proponents are right to point out that if this works for open source projects do you really need to put effort into other practices upstream of the PR? Opponents are also right to point out that adopting the work practice of an entirely different context into your salaried, work context is crazy. They are both right, individual organisations need to make deliberate decisions using the challenges of both side to the way they are working.

I’m not sure how much of this is a fightback by pairing advocates (that’s how I interpret this thread). There is a general feeling that pairing as a practice has declined.

In my work the practice is optional. As a manager I’ve always known that in terms of output delivery (which before people object, may be important particularly if you’re dealing with runway to meet salary) pairing is best when you’re increasing the average rather than facilitating the experienced.

I think even with pairing you’d want to do a code review step. Pairs are maybe better at avoid getting into weird approaches to solutions than an individual but they aren’t magical and if you are worried about dominant views and personalities pairing definitely doesn’t help solve that.

So I’d like to stick up for a few virtues of the Pull Request Review without making the argument that Pull Requests are all you need in your delivery process.

As an administrator of policy who often gets asked about Software Development LifeCycles (SDLC) as a required part of good software governance. It is handy to have a well documented, automated review process before code goes into production. It ticks a box at minimum disruption and there isn’t an alternative world where you’re just streaming changes into production on the basis of automated testing and production observability anyway.

As a maintainer of a codebase I rely on PRs a lot as documentation. Typically in support you handle issues on much more software than you have personally created. Pairing or mob programming isn’t going to work in terms of allowing me to support a wide ranging codebase. Instead it’s going to create demand for Level 3 support.

Well-structured PRs often allow me to understand how errors or behaviour in production relate to changes in code and record the intent of the people making the change. It makes it easier to see situations that were unanticipated or people’s conception of the software in use and how that varies from actual use.

PR review is also a chance for people outside the core developers to see what is happening and learn and contribute outside of their day to day work. Many eyes is not perfect but it is a real thing and people who praise teaching or mentoring as a way to improve technique and knowledge should be able to see that answering review questions (in a suitable form of genuine inquiry) is part of the same idea.

PRs also form a handy trigger for automated review and processes. Simple things like spelling checks allow a wider range of people to contribute to codebases fearlessly. Sure you don’t need a PR to use these tools but in my experience they seem more effective with the use of a stopping point for consideration and review of the work done to date.

Like a lot of things in the fashion-orientated, pendulum swinging world of software development good things are abandoned when no longer novel and are exaggerated to the point that they become harmful. It’s not a world known for thoughtful reflection and consideration. But saying that pull request reviews undermine trust and cohesion in teams or a formulaic practice without underlying benefit seems unhelpfully controversial and doctrinal.

Standard
Software, Work

The problem with developer job titles

Job titles are hard. This exchange on Twitter prompted a few thoughts that I couldn’t quite fit into a few smart-arsed twits.

In Chris’s Tweet he mentions that engineer is a co-opted title and that engineering is a discipline in its own right which most software groups don’t subscribe to because they aren’t really trying to do engineering. Not that this is a criticism but there is a massive difference between building a bridge or a road and creating a new web service. For a start there is a lot more established practice, science and understanding in physical engineering and more established and understood formal qualifications.

When I briefly worked in government helping create the Digital Careers framework people who were associated with Defence rightly objected to the confusion between software “engineering” and other engineers within professional frameworks. No-one is going to ask a developer to fix the combat damage on an airfield. I’ve previously joked that if we were honest we’d talk about “software overengineers” given that most developers struggle to find the simplest thing that works.

For the framework we settled on “developer” for people who wrote code and inconsistently used “engineering” for operational roles. I think on the basis that they created “infrastructure” where maybe the analogy makes a sort of sense.

I would also have gotten rid of “architect” if I’d had a chance; for exactly the same confusion but that term was too deeply embedded and still is a badge of prestige within the industry. Even now in the commercial world I have experienced hires wanting to be involved in “architecture” (and sadly not wanting to help me remodel my ground floor).

In Chris’s tweet he asks about what happened to the title “Programmer”. When I started in the industry this was indeed the coveted title and ideally I still think of myself this way even though it’s blatantly not true in the same way now.

However the issue with being a programmer is that jobs that literally involve just programming are few and far between. When I started in the industry the experienced developers were people who were at the tail-end of mainframe programming and a bit of what they were doing was still persuading machines to perform the tasks that were needed. The end was already in sight for pure programming jobs though. Some of my first professional programming work involved networking, a slightly dirty topic for the mainframe types.

Nowadays the emphasis is on understanding the domain space you are working on as well as the technical aspects of programming. I prefer the term “developer” (as others do) with the implication of being someone who develops systems of value via the medium of technology.

However that term also has its problems. When I worked at the Guardian I had a personal SEO battle with the Pune-based property development group for the search term “Guardian developers”. That battle seems to have been won now via sub-domain. This seems to be true more generally and now it is property developers who are having to use the prefix “property” on their job titles.

For a new profession not even past it’s first century, creating our professional lexicon is always going to be hard but in borrowing titles so shamelessly we are always creating problems for ourselves.

Programmer is probably the closest, truest name for what most of us do at the core of our role. For web developers though, assembling the typical bricolage of libraries and tooling is often an exercise in minimal programming and maximum duct taping. Perhaps it fairest to say that we are “software assemblers”, expect that might get confused with, you know, assemblers. Painful.

So in the end most of us are expected to bring capability in programming within teams that are creating technological systems of value. As long as programmers realise that programming is not the activity of value in itself then maybe we don’t need to worry so much about titles.

Standard
Web Applications

Roam Research: initial thoughts

Roam Research not only justified subscribing pretty much up front but has also made it onto my pinned tabs in virtually no time flat. It’s basically a web-based knowledge management system. I’m already a fan of Workflowy so I’m already comfortable with putting information into trees and hierarchies, in fact there’s a lot of overlap between the two applications as you can just use Roam as a kind of org-mode bulleted list organiser.

The thing that makes it different is the ability to overlay a wiki-like ability to turn any piece of text into a link which creates another list page to store other notes.

The resulting page highlights the linked portions of the trees in other pages as well as containing it’s own content.

The links then form a graph that can be explored but I haven’t generate enough content for it to be generating any useful insight yet.

The pages are searchable so you can either take wiki-like journeys of discovery through your notes or just search and jump to anything relevant in your knowledge graph.

By default the system creates a daily “diary” page for you to record notes in an initially unstructured way organically as you roll through the day. I’m still primarily in my todo lists in a Getting Things Done mode during the day but I have found it a useful end of day technique for reflecting or summarising ideas to follow up on.

Roam is very much influenced by and part of the new wave of knowledge management tools based on Zettelkasten. If you’re unfamiliar it’s worth reading up on it (I don’t know it well enough to create a pithy summary).

To date though everything I’ve tried in this space was a bit formal and tricky to get going or fit into my existing ways of working. Roam on the other hand is web-based, relatively quick and usable and uses enough metaphors from existing systems to make it feel accessible.

Weirdly the first use that convinced me I needed this service was actually recipes. You can have a hierarchy of different types of recipes but use a link and you can have a vertical slice across ingredients or techniques.

The second was while genuinely doing some market research on Javascript enhancement frameworks where I wanted to have one page for my overall thoughts (“Is this something to pursue?”) and was able to break the list of all the frameworks I was looking at into their own pages with links to the frameworks and any thoughts I had as I was playing around with them.

The mobile experience isn’t quite as good, it’s a kind of fast noting system where I’m not sure how I can quickly attach a thought to an existing page. Here it’s still easier to use a note-taking app and consolidate thoughts later.

Overall though this is still the most exciting web app I’ve used this year.

Standard