Category Archives: Software

Software

Futurespectives

Futurespectives seem to be a much rarer practice than retrospectives. I learnt about using Futurespectives when I working with ThoughtWorks and I’ve used them a few times but I can’t seem to find a great online resource to introduce people to the idea. Liz Keogh’s advice on Futurespectives is probably the best I’ve found (beyond a lot of retro as a service companies writing marketing blog material about them).

One reason for their lack of adoption is that they require a certain amount of speculative imagination and that sometimes doesn’t come easily to developers who are very rooted in the realities of their work and sometimes think it is fanciful to speculate about the future.

However if you can persuade people to engage then I find this exercise to be very valuable for surfacing concerns and getting delivery teams to align on the broad shape of their approach to the work. It often sparks conversations that are being suppressed particular if people are being pressed for “commitments” on the upcoming work.

As with retrospectives, generating responses to the initial questions is best done independently and the consolidation of the individual answers and the discussion of what they reveal is best done collectively.

I ask the following questions but as with all practices it is often worth investing some time in trying to figure out what the purpose of the exercise is and what questions would best elicit responses that drive the conversation forward.

As a facilitator the frame for these questions are: “Imagine that we have completed our project. It has been a success even if at times it may have been hard work. The project meets the requirements and is working well in production. Our solution may be different from what we imagine today but we were able to adopt new ideas successfully. The team is happy and satisfied with how the work has gone and we didn’t need to make any excessive requests on their time and skills.”

  • How were we successful?
  • What problems did we have to overcome?
  • What are we proud about what we’ve done?

I ask people to generate responses from their perspective alone although they are free to speculate about how other teams and people will have helped or contributed along the way.

If people are struggling with the exercise I sometimes try to provide some starting questions. How do you feel about the project being complete? What do you feel satisfied to have done? What went better than you were expecting? How do other people feel about the work when they are talking to you about it?

Again the frame for all these questions is that the project has been successful (despite any doubts the participant may have now); the engineering mindset needs to accept that as definite thing and that the problem to be solved is: how was it successful despite this doubts? How were the problems solved or mitigated?

This last part is the critical step because it typically allows people to apply unconventional problem solving ideas. Typically people who are worried about a future problem cannot get past it if they feel it is unsurmountable however if you tell them that someone else has already solved it then just knowing that a solution exists allows people to reframe the problem and overcome their block on what the answer may be.

During the consolidation phase of the exercise, you bring the individual answers together and play them back to the group as a whole. This element is exactly the same as facilitating a regular retrospective. Try and ensure that any explanation of people’s ideas that the group needs is done during this phase. Often people are more aligned than they think but if there are any sharp disagreements in the approach they will typically come out now and its important that participants don’t reject any ideas at this stage because they will just return to their existing mindset.

Pay particular attention to similar ideas using different language, this can indicate that people are probably approaching the problems in a similar way but aren’t yet communicating enough to have shared ideas or a collaborative design approach. If there’s a lot of this it may be worth setting up a follow up to just review and consolidate the current state of play in the project. It may be that preparation is being rushed and the team isn’t having enough time to work together.

After creating and consolidating the initial input we now look at the three questions in a different way to help us generate actions from the futurespective. I sum up how we move from our imagined future to actions today in the following way for each question:

  • How do we realise our expected paths to success? What needs to happen to start towards that outcome? (Make true)
  • It is likely that we will encounter our anticipated problems, how can we minimise the impact they will have on us? (De-risk)
  • How can we ensure we have pride in our work? (Achieve)

At this point the session is more of a facilitated free-for-all with the initial phase being open to all ideas and suggestions. Some really common actions are that technical leaders realise they need to share more information on their vision and ideas with the rest of the team. It is also really common that when several people anticipate the same problem that prototyping, testing or training can be done very early on in the project plan to remove the problem or shift it a better understood class of problem.

The pride question often has actions that are associated with process, quality and shared standards and beliefs. Often though ideas about collaboration and the “team contract” come into play. Leaders can explain what others can rely on them for and what they want from the rest of the team. People can share fears in a way that allows people in authority to acknowledge that they share the same fears in a safe way. The format encourages not just the expression of fear but how will we manage our anxiety about the upcoming work.

In many ways if you’ve facilitated a retrospective you have all the skills that are required to run a futurespective, the tricky thing is about getting the participants in the right frame of mind.

In terms of measuring the impact of a successful futurespective you should be able to see a move from analysis to action and a growth of shared language and outcomes. Perceptions between the key participants of the project should be positive as they are already imagining a successful partnership ahead.

Standard
Programming, Software, Work

Defining the idea of “software engineering”

I have been reading Dave Farley’s Modern Software Engineering. Overall it’s a great read and thoroughly recommended (I’m still reading through it but I’ve read enough to know it is really interesting and a well-considered approach to common problems in development).

One of the challenges Dave tackles is to try and provide a definition of what software engineering actually is. This is actually a pretty profound challenge in my view. I’ve often felt that developers have usurped the title of engineer to provide a patina of respectability to their hacky habits. Even in Dave’s telling of the origin of the term it was used to try and provide parity of esteem with hardware engineers (by no lesser figure than Margaret Hamilton).

In large organisations where they have actual engineers it is often important to avoid confusion between what Dave categorises as Design and Production engineering. Software engineering sits in the world of design engineering. Software is malleable and easy to change unlike a supply chain or a partially completed bridge. Where the end result of the engineering process is an expensive material object Dave points that it is common to spend a lot of time modelling the end result and refining the delivery process for the material output as a result of the predictions of the model. For software to some extent our model is the product and we can often iterate and refine it at very low cost.

Dave proposes the following definition of engineering:

Engineering is the application of an empirical, scientific approach to finding efficient, economical solutions to practical problems.

Dave Farley, Modern Software Engineering

This definition is one I can live with and marries my experience of creating software to the wider principles of engineering. It also bridges across the two realms of engineering, leaving the differences to practices rather than principles.

It is grounded in practicality rather than aloof theories and it emphasises that capacities drive effective solutions as much as needs. This definition is a huge step forward in being able to build consensus around the purpose of a software engineer.

Standard
Software

State of the Browser 2021

This is the first in-person conference I’ve been to since the pandemic and since it normally clashes with PyCon UK this is also the first State of the Browser that I’ve been too in a while.

As a high-level pitch for the conference it a chance to hear from standard makers and browser developers about their thoughts on the web, web standards and issues in web development.

The conference had an audience of probably a third of what I felt it had the last time I attended in person. There was not issue with distancing and you could add a stickers to your attendee batch to nix photography and to ask for people to keep their distance.

Usually the chance to socialise and network is a major part of the conference experience but once I was there I realised that I didn’t really want to spend the time required to get to know someone new while COVID is as prevalent as it is, not attend the generous post-conference drinks.

Which made me wonder why I was there at all. The answer, on reflection, is that being physically present meant that I was actually present for the talks as well. I’ve bought tickets for virtual events earlier in the year and I still haven’t watch the videos.

By physically turning up I did pay more attention and I did engage and learn more than I did virtually.

I found a few things about the conference frustrating though. Firstly a number of the speakers weren’t there and instead had recorded a talk so being at the conference ended up being a collective video watch without being able to control the video and skip the boring bits. Also there were no questions from the audience because that was being handled on Discord. Now most of my Discord is taken up with gaming because, y’know that’s what Discord pretty much is for the most part. So I wasn’t able to see that side of things because I didn’t have time to set up some kind of work account. But generally whether it was Slack or something else I kind of think having the questions on the conference chat meant that the talks were actually lectures and where the speakers weren’t that proficient with their delivery it made the talks more boring.

So at the end of the experience I have no idea as to whether my attendance was a good idea or not. I probably would have been distracted at home but at least I could have sorted out Discord and have watched the pre-recorded videos in a more comfortable environment (I certainly could of dodged the morning torrential rain).

But when there was a good in-person speaker it was great. Rachel Andrew was the standout for managing a review of the history of layout systems while also previewing the thinking of the standards groups. In particular drawing a fascinating line between the necessity of the contains CSS directive to the ability to be able to look forward to container queries. Stephanie Stimac shared similar insight into what the future may hold for the development of the Form elements and their backwards-compatible codification and customisation.

Alex Russell offered a rebuttal of the locked down mobile ecosystems from a capitalist perspective but failed to really offer remedies given that this overall is a capitalist failure.

In a totally different vibe Heydon Pickering did a talk about requiring people to switch off Javascript to read his blog. It was closer to standup and I did laugh out loud several times although trying to explain what made it funny and entertaining has proven highly difficult.

Rachel Andrew is one of the people behind Notist which a few people were using to share slide links. I hadn’t heard of it before and I can see it’s pretty handy compared to trawling Youtube trying to figure out if some talk you half remember has been posted there.

Overall I think it was worth the effort, I felt I got outside my bubble for a while and felt a bit more connected to the efforts that are still ongoing to safeguard and advance the web as a whole.

Standard
Software, Work

The problem with developer job titles

Job titles are hard. This exchange on Twitter prompted a few thoughts that I couldn’t quite fit into a few smart-arsed twits.

In Chris’s Tweet he mentions that engineer is a co-opted title and that engineering is a discipline in its own right which most software groups don’t subscribe to because they aren’t really trying to do engineering. Not that this is a criticism but there is a massive difference between building a bridge or a road and creating a new web service. For a start there is a lot more established practice, science and understanding in physical engineering and more established and understood formal qualifications.

When I briefly worked in government helping create the Digital Careers framework people who were associated with Defence rightly objected to the confusion between software “engineering” and other engineers within professional frameworks. No-one is going to ask a developer to fix the combat damage on an airfield. I’ve previously joked that if we were honest we’d talk about “software overengineers” given that most developers struggle to find the simplest thing that works.

For the framework we settled on “developer” for people who wrote code and inconsistently used “engineering” for operational roles. I think on the basis that they created “infrastructure” where maybe the analogy makes a sort of sense.

I would also have gotten rid of “architect” if I’d had a chance; for exactly the same confusion but that term was too deeply embedded and still is a badge of prestige within the industry. Even now in the commercial world I have experienced hires wanting to be involved in “architecture” (and sadly not wanting to help me remodel my ground floor).

In Chris’s tweet he asks about what happened to the title “Programmer”. When I started in the industry this was indeed the coveted title and ideally I still think of myself this way even though it’s blatantly not true in the same way now.

However the issue with being a programmer is that jobs that literally involve just programming are few and far between. When I started in the industry the experienced developers were people who were at the tail-end of mainframe programming and a bit of what they were doing was still persuading machines to perform the tasks that were needed. The end was already in sight for pure programming jobs though. Some of my first professional programming work involved networking, a slightly dirty topic for the mainframe types.

Nowadays the emphasis is on understanding the domain space you are working on as well as the technical aspects of programming. I prefer the term “developer” (as others do) with the implication of being someone who develops systems of value via the medium of technology.

However that term also has its problems. When I worked at the Guardian I had a personal SEO battle with the Pune-based property development group for the search term “Guardian developers”. That battle seems to have been won now via sub-domain. This seems to be true more generally and now it is property developers who are having to use the prefix “property” on their job titles.

For a new profession not even past it’s first century, creating our professional lexicon is always going to be hard but in borrowing titles so shamelessly we are always creating problems for ourselves.

Programmer is probably the closest, truest name for what most of us do at the core of our role. For web developers though, assembling the typical bricolage of libraries and tooling is often an exercise in minimal programming and maximum duct taping. Perhaps it fairest to say that we are “software assemblers”, expect that might get confused with, you know, assemblers. Painful.

So in the end most of us are expected to bring capability in programming within teams that are creating technological systems of value. As long as programmers realise that programming is not the activity of value in itself then maybe we don’t need to worry so much about titles.

Standard
Programming, Software, Web Applications, Work

Prettier in anger

I’ve generally found linting to be a pretty horrible experience and Javascript/ES haven’t been any exception to the rule. One thing that I do agree with the Prettier project is that historically linters have tried to perform two tasks to mixed success: formatting code to conventions and performing static analysis.

Really only the latter is useful and the former is mostly wasted cycles except for dealing with language beginners and eccentrics.

Recently at work we adopted Prettier to avoid having to deal with things like line-lengths and space-based tab sizes. Running Prettier over the codebase left us with terrible-looking cramped two-space tabbed code but at least it was consistent.

However having started to live with Prettier I’ve been getting less satisfied with the way it works and Prettier ignore statements have been creeping into my code.

The biggest problem I have is that Prettier has managed its own specific type of scope creep out of the formatting space. It rewrites way too much code based on line-size limits and weird things like precedent rules in boolean statements. So for example if you have a list with only one entry and you want to place the single entry on a separate line to make it clear where you intend developers to extend the list Prettier will put the whole thing on a single line if it fits.

If you bracket a logical expression to help humans parse the meaning of the statements but the precedent rules mean that brackets are superfluous then Prettier removes them.

High-level code is primarily written for humans, I understand that the code is then transformed to make it run efficiently and all kinds of layers of indirection are stripped out at that point. Prettier isn’t a compiler though, it’s a formatter with ideas beyond its station.

Prettier has also benefited from the Facebook/React hype cycle so we, like others I suspect, are using it before it’s really ready. It hides behind the brand of being “opinionated” to avoid giving control over some of its behaviour to the user.

This makes using Prettier a kind of take it or leave it proposition. I’m personally in a leave it place but I don’t feel strongly enough to make an argument to remove from the work codebase. For me currently tell Prettier to ignore code, while an inaccurate expression of what I want it to do, is fine for now while another generation of Javascript tooling is produced.

Standard
Software

Passive-aggressive collaboration

One interesting (and depressing) aspect of post-Github open source development is the use of the pull request as a passive aggressive way of putting off potential contributors, users and testers.

I have had the experience of discovering an issue with a piece of open source and raising an issue for it only to be “invited” to contribute a reliable failing test case, with a fix and all to the project’s contribution standards.

Now the joy of open source is being able to scratch your itch and prioritising your own problems by contributing solutions.

However there are often very good reasons why the maintainers should be the ones fixing the issues.

Firstly the maintainers should be the ones who gain most from fixing issues. One cool thing about Git-based development is that forking allows you to use existing codebases but not share the views and priorities of the original project. If I disagree with the direction or design of a project I can fork it and completely change the code to match my own aesthetics and priorities.

However in most cases I agree with the direction of the maintainers and I am simply pointing out an issue or problem that maybe they haven’t encountered in their context. I could scratch my itch but it is often more effective if the maintainer took my use case into consideration and reworked the codebase to include it.

Maintainers:

  • have more context on the code base
  • know more about the problems the code tackles
  • know their conventions and coding standards better than I do
  • have more invested in having an effective solution than I do

Looking at things like Guava which while open are effectively not open to contribution. This is a more honest approach than inviting non-trivial contributions.

Trying to take the perspective of the maintainer I know it is tedious when people do things outside of your area of interest (i.e. IE fixes when you’re not targeting that version of the browser or Windows fixes in a project targeted at UNIXes). However telling people to “fix it themselves” is not as honest as saying “that’s not our focus”. People can then decide whether they want to fork or not.

Standard
Software

In praise of fungible developers

The “fungibility” of developers is a bit of hot topic at the moment. Fungibility means the ability to substitute one thing for another for the same effect; so money is fungible for goods in modern economies.

In software development that means taking a developer in one part of the organisation and substituting them elsewhere and not impacting the productivity of either developer involved in the exchange.

This is linked to the mythical “full-stack” developer by the emergence of different “disciplines” within web software development, usually these are: devops, client-side (browser-based development) and backend development (services).

It is entirely possible for developers to enter one of these niches and spend all their time in it. In fact sub-specialisations in things like responsive CSS and single-page apps (SPA) are opening up.

Now my view has always been that a developer should always aspire to have as broad a knowledge base as possible and to be able to turn their hand to anything. I believe when you don’t really understand what is going on around your foxhole then problems occur. Ultimately we are all pushing electric pulse-waves over wires and chips and it is worth remembering that.

However my working history was pretty badly scarred by the massive wave of Indian outsourcing that happened post the year 2000 and as a consequence the move up the value-chain that all the remaining onshore developers made. Chad Fowler’s book is a pretty good summary of what happened and how people reacted to it.

For people getting specialist pay for niche work, full-stack development doesn’t contain much attraction. Management sees fungibility as a convenient way of pushing paper resources around projects and then blaming developers for not delivering. There are also some well-written defences of specialisation.

In defence of broad skills

But I still believe that we need full-stack developers and if you don’t like that title then let’s call them holistic developers.

Organisations do need fungibility. Organisations without predictable demand or who are experiencing disruption in their business methodology need to be flexible and they need to respond to situations that are unexpected.

You also need to fire drill those situations where people leave, fall ill or have a family crisis. Does the group fall apart or can it readjust and continue to deliver value? In any organisation you never know when you need to change people round at short notice.

Developers with a limited skill set are likely to make mistakes that someone with a broader set of experiences wouldn’t. It is also easier for a generalist developer to acquire specialist knowledge when needed than to broaden a specialist.

Encouraging specialism is the same as creating knowledge silos in your organisation. There are times when this might be acceptable but if you aren’t doing it in a conscious way and accompanying it with a risk assessment then it is dangerous.

Creating holistic developers

Most organisations have an absurd reward structure that massively benefits specialists rather than generalists. You can see that in iOS developer and mobile responsive web CSS salaries. The fact that someone is less capable than their colleagues means they are rewarded more. This is absurd and it needs to end.

Specialists should be treated like contractors and consultants. They have special skills but you should be codifying their knowledge and having them train their generalist colleagues. A specialist should be seen as a short-term investment in an area where you lack institutional memory and knowledge.

All software delivery organisations should practice rotation. Consider it a Chaos Monkey for your human processes.

Rotation puts things like onboarding processes to the test. It also brings new eyes to the solution and software design of the team. If something is simple it should make sense and be simply to newcomer, not someone who has been on the team for months.

Rotation applies within teams too. Don’t give functionality to the person who can deliver it the fastest, give it to the person who would struggle to deliver it. Then force the rest of the team to support that person. Make them see the weaknesses in what they’ve created.

Value generalists and go out of your way to create them.

Standard
Software

The sly return of Waterfall

No-one does Waterfall any more of course, we’re all Agile incrementalists. It is just that a lot of things are difficult to tackle in increments. You can’t get a great design, for example, or a visual style guide without a lot of user testing and workshopping. From a technical perspective you need to make sure your scaling strategy is baked in from the start and to help support that you will also want to have a performance testing framework in place. You’ll also want to be running those test suites in a continuous deployment process, because its hard to create that after the fact.

In short apart from the actual software you want to do everything else up front.

Waterfall existed for a reason, it tried to fix certain issues with software development, it made sure that when you finished a step in the process you didn’t have to go back and re-visit it. It made you think about all the issues that you would encounter in creating complex software and come up with a plan for dealing with them.

Therefore I can see all the exciting enticements to make sure you do something “up front because it will be difficult to solve later”.

However Waterfall had to change because it didn’t work and dressing up the same process in the guise of sensible forethought doesn’t make it more viable than its predecessors.

It can be really frustrating to have a product that looks ugly, or is slow or constantly falls over. It is far more frustrating to have a stable, beautiful and irrelevant product.

Occasionally you know that a product is going to take off like a rocket, as with fast-follow products for example, and it is going to fully payback a big investment in its creation. However in all other cases you have no idea whether a product is going to work and therefore re-coup its investment.

Even with an existing successful product major changes to the product are just as likely to have unexpected consequences as they are to deliver expected benefits. Sometimes they do both.

What always matters is the ability to change direction and respond to the circumstances that you find yourself in. Some aspects of software development have seen this and genuinely try to implement it. Other parts of the discipline are engaged in sidling back into the comfortable past under the guise of “responsibility”.

Standard
Software

Switching Nvidia drivers from the command-line

The Steam client informed me today that there were more recent Nvidia drivers for Ubuntu available and that I should upgrade for stability, etc. etc. It seemed a fairly innocuous change compared to the beta drivers I was using so I pressed the button and then restarted. Resulting in a failure to boot Unity and a chance to rediscover the joy of the command-line.  I don’t know why Unity and the new drivers fail to mix to spectacularly however the simplest thing to do seemed to be to revert to the earlier drivers.

The problem with doing that is that I’ve only ever done it via the gui tools. This AskUbuntu answer told me about Jockey the software that underpins the proprietary driver contol. Running Jockey at the command-line was very, very slow but it did indeed allow me to select the earlier drivers and after a restart the GUI was booting again. Much easier than hand-editing an X config file.

Standard
Software, Work

Up-front quality

There has been a great exchange on the London Clojurians mailing list recently talking about the impact of a good REPL on development cycles. The conversation kicks into high-gear with this post from Malcolm Sparks although it is worth reading it from the start (membership might be required I can’t remember). In his post Malcolm talks about the cost of up-front quality. This, broadly speaking, is the cost of the testing required to put a feature live, it is essentially a way of looking at the cost that automated testing adds to the development process. As Malcolm says later: “I’m a strong proponent of testing, but only when testing has the effect of driving down the cost of change.”.

Once upon a time we had to fight to introduce unit-testing and automated integration builds and tests. Now it is a kind of given that this is a good thing, rather like a pendulum, the issue is going too far in the opposite direction. If you’ve ever had to scrap more than one feature because it failed to perform then the up-front quality cost is something you consider as closely as the cost of up-front design and production failure.

Now the London Clojurians list is at that perfect time in its lifespan where it is full of engaged and knowledgeable technologists so Steve Freeman drops into the thread and sensibly points out that Malcolm is also guilty of excess by valuing feature mutability to the point of wanting to be able to change a feature in-flight in production, something that is cool but is probably in excess of any actual requirements. Steve adds that there are other benefits to automated testing, particularly unit testing, beyond guaranteeing quality.

However Steve mentions the Forward approach, which I also subscribe to, of creating very small codebases. So then Paul Ingles gets involved and posts the best description I’ve read of how you can use solution structure, monitoring and restrained codebases to avoid dealing with a lot of the issues of software complexity. It’s hard to boil the argument down because the post deserves reading in full. I would try and summarise it as the external contact points of a service are what matters and if you fulfil the contract of the service you can write a replacement in any technology or stack and put the replacement alongside the original service.

One the powerful aspects of this approach is that is generalises the “throw one away” rule and allows you to say that the current codebase can be discarded whenever your knowledge of the domain or your available tools change sufficiently to make it possible to write an improved version of the service.

Steve then points out some of the other rules that make this work, being able to track and ideally change consumers as well. Its an argument for always using keys on API services, even internal ones, to help see what is calling your service. Something that is moving towards being a standard at the Guardian.

So to summarise, a little thread of pure gold and the kind of thing that can only happen when the right people have the time to talk and share experiences. And when it comes to testing, ask whether your tests are making it cheaper to change the software when the real functionality is discovered in production.

Standard