Tag Archives: software delivery

London, Work

Will humans still create software?

Recently I attended the London CTOs Unconference: an event where senior technical leaders discuss various topics of the day.

There were several proposed sessions on AI and its impact on software delivery and I was party of a group discussing the evolution of AI-assisted development and looking at the how this would change and ultimately what role people thought there would be for humans in software delivery (we put the impacts of artificial general intelligence to one side to focus on what happens with the technologies we currently have).

The session was conducted under Chatam House equivalent rules so I’m just going to record some of the discussion and key points in this post.

Session notes

Currently we are seeing automating of existing processes within the delivery lifecycle but there are opportunities to rethink how we deliver software that makes better use of the current generative AI tools and perhaps sets us up to take advantage of better options in future.

Rejecting a faster horse

Thinking about the delivery of the whole system rather than just modules of code, configuration or infrastructure allows us to think about setting a bigger task than simply augmenting a human. We can start to take business requirements in natural language, generate product requirement documents from these and then use formal methods to specify the behaviour of the system and verify that the resulting software that generative AI creates meets these requirements. Some members of the group had already been generating such systems and felt it was a more promising approach than automating different roles in the current delivery processes.

Although these methods have existed for a while they are not widely used currently and therefore it seems likely that reskilling will be required by senior technical leaders. Defining the technical outcomes they are looking for through more formal structures that work better with machines requires both knowledge and skill. Debugging is likely to move from the operation of code to the process of generation within the model and leading to an iterative cycle of refinement of both prompts and specifications. In doing this, the recent move to expose more chain of thought information to user is helpful and allows the user to refine their prompt when they can see flaws in the reasoning process of the model.

The fate of code

We discussed whether code would remain the main artefact of software production and we didn’t come to a definite conclusion. The existing state of the codebase can be given as a context to the generation process, potentially refined as an input in the same way as retrieval augmented generation works.

However if the construction of the codebase is fast and cheap then the value of code retention was not clear, particularly if requirements or specifications are changing and therefore an alternative solution might be better than the existing one.

People experimenting with whole solution generation do see major changes between iterations; for example where the model selects different dependencies. For things like UIs this matters in terms of UX but maybe it doesn’t matter so much for non-user facing things. If there is a choice of database mappers for example perhaps we only care that the performance is good and that SQL injection is not possible.

Specifications as artefacts

Specifications and requirements need to be versioned and change controlled exactly as source code is today. We need to ensure that requirements are consistent and coherent, which formal methods should provide, but analysis and resolution of differing viewpoints as to the way system works will remain an important technical skill.

Some participants felt that conflicting requirements would be inevitable and that it would be unclear as to how the generating code would respond to this. It is certainly clear that currently models do not seem to be able to identify the conflict and will most probably favour one of the requirements over the others. If the testing suite is independent than behavioural tests may reveal the resulting inconsistencies.

It was seen as important to control your own foundation model rather than using external services. Being able to keep the model consistent across builds and retain a working version of it decouples you from vendor dependencies and should be considered as part of the build and deployment infrastructure. Different models have different strengths (although some research contradicts this anecdotal observation). We didn’t discuss supplementation techniques but we did talk about priming the code generation process with coding standards or guidelines but this did not seem to be a technique currently in use.

For some participants using generative AI was synonymous with choosing a vendor but this is risky as one doesn’t control the lifespan of such API-based interactions or how a given model might be presented by the vendor. Having the skills to manage your own model is important.

In public repositories it has been noted that the volume of code produced has risen but quality has fallen and that there is definitely a trade-off being made between productivity and quality.

This might be different in private codebases where different techniques are used to ensure the quality of the output. People in the session trying these techniques say they are getting better results than what is observed in public reporting. Without any way of verifying this though people will just have to experiment for themselves and see if they can improve on the issues seen in the publicly observed code.

When will this happen in banks?

We talked a little bit about the rate of adoption of these ideas. Conservative organisations are unlikely to move on this until there is plenty of information available publicly. However if an automated whole system creation process works there are significant cost savings associated with it and projects that would previously have been ruled out as too costly become more viable.

What does cheap, quick codebases imply?

We may be able to retire older systems with a lot of embedded knowledge in them far more quickly than if humans had to analyse, extract and re-implement that embedded knowledge. It may even be possible to recreate a mainframe’s functionality on modern software and hardware and make it cheaper than training people in COBOL.

If system generation is cheap enough then we could also ask the process to create implementations with different constraints and compare different approaches to the same problem optimising for cost, efficiency or maintenance. We can write and throw away many times.

What about the humans?

The question of how humans are involved in the software delivery lifecycle, what they are doing and therefore what skills they need was unclear to us. However no-one felt that humans would have no role in software development but that it was likely to be different to the skill set the made people successful today.

It also seemed unlikely that a human would be “managing” a team of agents if the system of specification and constraints was adopted. Instead humans would be working at a higher level of abstraction with a suite of tools to deliver the implementation. Virtual junior developers seemed to belong to the faster horse school of thinking.

Wrapping up the session

The session lasted for pretty much the whole of the unconference and the topic often went broad which meant there were many threads and ideas that were not fully resolved. It was clear that there are currently at least two streams of experimentation: supplementing existing human roles in the software delivery cycle with AI assistance and reinventing the delivery process based on the possibilities offered by cheap large language models.

As we were looking to the future we most discussed the second option and this seems to be what people have in mind when they talk about not needing experienced software developers in future.

In some ways this is the technical architect’s dream in that you can start to work with pure expressions of solutions to problems that are faithfully adhered to by the implementer. However the solution designer now needs to understand how and why the solution generation process can go wrong and needs to verify the correctness and adherence of the final system. The non-deterministic nature of large language models are not going away and therefore solution designers need to think carefully about their invariants to ensure consistency and correctness.

There was a bit of an undertow in our discussions about whether it was a positive that a good specification almost leads to a single possible solution or whether we need to allow the AI to confound our expectation of the solution and created unexpected things that met our specifications.

The future could be a perfect worker realising the architect’s dream or it could be more like a partnership where the human is providing feedback on a range of potential solutions provided by a generation process, perhaps with automated benchmarking for both performance and cost.

It was a really interesting discussion and an interesting snapshot of what is happening in what feels like an area of frenzied activity from early adopters, later adopters probably can afford to give this area more time to mature as long as they keep an eye on whether the cost of delivery is genuinely dropping in the early production projects.

Standard
Software

Great software delivery newsletters

I currently subscribe to a number of great newsletters around technology and software delivery. While the Fediverse is also a great place to pick up news and gossip I have found that there is something really valuable in having a regular curated round up of interesting articles. It may be no surprise that the consistently great newsletters are produced by people who are engaged in consultancy. I think they inevitably get exposed to trends and concerns in the industry and also can commit the time to writing up their thoughts and reflecting on their chosen content.

Pat Kua‘s Level Up focuses on technical leadership and tends to have good pieces around human factors, managing yourself and creating good systems for delivery. It also often has advice pieces for people coming into technical management or leadership.

John Cutler’s The Beautiful Mess focuses on Product but is also great on strategy and importantly is always focused on getting to a better product process by emphasising collaboration and breaking down barriers between functional silos. I also enjoy reading how he approaches putting together alternatives to roadmaps and strategy documents. I think he has the best sense on how to use things like metrics and North Stars.

Emily Weber writes Posts from Awesome Folk has a focus on management, leadership, consensus building and healthy organisation cultures. As the title suggests it offers a carefully curated selection of posts that are often longer form and are generally from expert practitioners.

Michael Brunton-Spall‘s Cyber Weekly is your one stop shop for news on security and analysis of the key issues of the day.

Simon Willison‘s newsletter is more recent and feels more like a very long blog that is getting pushed into the newsletter format. Despite this Simon is one of the most creative and independent developers you could read and he was early into the LLM and generative AI and has lots of interesting insight into what you can do with these models and what works and what doesn’t. He’s also an (intimidating) role model for what independent, solo devs can achieve.

I have a lot of other subscriptions (and indeed a lot of people seem to be starting newsletters currently) so I will probably need to do a follow up to this post in a couple of months if I see that people are posting consistently useful things. One general thing to point out is that if I’m working on a particular technology (like Django, Go or React) I’ll often subscribe to the weekly community news roundups to get a feel for what’s happening. However I find the volume of links and items is overwhelming if you don’t have a specific interest or purpose in reading through them so I relegate them to RSS when I’m not actively working with them and have a more occasional catchup.

Standard
Software

Futurespectives

Futurespectives seem to be a much rarer practice than retrospectives. I learnt about using Futurespectives when I working with ThoughtWorks and I’ve used them a few times but I can’t seem to find a great online resource to introduce people to the idea. Liz Keogh’s advice on Futurespectives is probably the best I’ve found (beyond a lot of retro as a service companies writing marketing blog material about them).

One reason for their lack of adoption is that they require a certain amount of speculative imagination and that sometimes doesn’t come easily to developers who are very rooted in the realities of their work and sometimes think it is fanciful to speculate about the future.

However if you can persuade people to engage then I find this exercise to be very valuable for surfacing concerns and getting delivery teams to align on the broad shape of their approach to the work. It often sparks conversations that are being suppressed particular if people are being pressed for “commitments” on the upcoming work.

As with retrospectives, generating responses to the initial questions is best done independently and the consolidation of the individual answers and the discussion of what they reveal is best done collectively.

I ask the following questions but as with all practices it is often worth investing some time in trying to figure out what the purpose of the exercise is and what questions would best elicit responses that drive the conversation forward.

As a facilitator the frame for these questions are: “Imagine that we have completed our project. It has been a success even if at times it may have been hard work. The project meets the requirements and is working well in production. Our solution may be different from what we imagine today but we were able to adopt new ideas successfully. The team is happy and satisfied with how the work has gone and we didn’t need to make any excessive requests on their time and skills.”

  • How were we successful?
  • What problems did we have to overcome?
  • What are we proud about what we’ve done?

I ask people to generate responses from their perspective alone although they are free to speculate about how other teams and people will have helped or contributed along the way.

If people are struggling with the exercise I sometimes try to provide some starting questions. How do you feel about the project being complete? What do you feel satisfied to have done? What went better than you were expecting? How do other people feel about the work when they are talking to you about it?

Again the frame for all these questions is that the project has been successful (despite any doubts the participant may have now); the engineering mindset needs to accept that as definite thing and that the problem to be solved is: how was it successful despite this doubts? How were the problems solved or mitigated?

This last part is the critical step because it typically allows people to apply unconventional problem solving ideas. Typically people who are worried about a future problem cannot get past it if they feel it is unsurmountable however if you tell them that someone else has already solved it then just knowing that a solution exists allows people to reframe the problem and overcome their block on what the answer may be.

During the consolidation phase of the exercise, you bring the individual answers together and play them back to the group as a whole. This element is exactly the same as facilitating a regular retrospective. Try and ensure that any explanation of people’s ideas that the group needs is done during this phase. Often people are more aligned than they think but if there are any sharp disagreements in the approach they will typically come out now and its important that participants don’t reject any ideas at this stage because they will just return to their existing mindset.

Pay particular attention to similar ideas using different language, this can indicate that people are probably approaching the problems in a similar way but aren’t yet communicating enough to have shared ideas or a collaborative design approach. If there’s a lot of this it may be worth setting up a follow up to just review and consolidate the current state of play in the project. It may be that preparation is being rushed and the team isn’t having enough time to work together.

After creating and consolidating the initial input we now look at the three questions in a different way to help us generate actions from the futurespective. I sum up how we move from our imagined future to actions today in the following way for each question:

  • How do we realise our expected paths to success? What needs to happen to start towards that outcome? (Make true)
  • It is likely that we will encounter our anticipated problems, how can we minimise the impact they will have on us? (De-risk)
  • How can we ensure we have pride in our work? (Achieve)

At this point the session is more of a facilitated free-for-all with the initial phase being open to all ideas and suggestions. Some really common actions are that technical leaders realise they need to share more information on their vision and ideas with the rest of the team. It is also really common that when several people anticipate the same problem that prototyping, testing or training can be done very early on in the project plan to remove the problem or shift it a better understood class of problem.

The pride question often has actions that are associated with process, quality and shared standards and beliefs. Often though ideas about collaboration and the “team contract” come into play. Leaders can explain what others can rely on them for and what they want from the rest of the team. People can share fears in a way that allows people in authority to acknowledge that they share the same fears in a safe way. The format encourages not just the expression of fear but how will we manage our anxiety about the upcoming work.

In many ways if you’ve facilitated a retrospective you have all the skills that are required to run a futurespective, the tricky thing is about getting the participants in the right frame of mind.

In terms of measuring the impact of a successful futurespective you should be able to see a move from analysis to action and a growth of shared language and outcomes. Perceptions between the key participants of the project should be positive as they are already imagining a successful partnership ahead.

Standard
Programming, Software, Work

Defining the idea of “software engineering”

I have been reading Dave Farley’s Modern Software Engineering. Overall it’s a great read and thoroughly recommended (I’m still reading through it but I’ve read enough to know it is really interesting and a well-considered approach to common problems in development).

One of the challenges Dave tackles is to try and provide a definition of what software engineering actually is. This is actually a pretty profound challenge in my view. I’ve often felt that developers have usurped the title of engineer to provide a patina of respectability to their hacky habits. Even in Dave’s telling of the origin of the term it was used to try and provide parity of esteem with hardware engineers (by no lesser figure than Margaret Hamilton).

In large organisations where they have actual engineers it is often important to avoid confusion between what Dave categorises as Design and Production engineering. Software engineering sits in the world of design engineering. Software is malleable and easy to change unlike a supply chain or a partially completed bridge. Where the end result of the engineering process is an expensive material object Dave points that it is common to spend a lot of time modelling the end result and refining the delivery process for the material output as a result of the predictions of the model. For software to some extent our model is the product and we can often iterate and refine it at very low cost.

Dave proposes the following definition of engineering:

Engineering is the application of an empirical, scientific approach to finding efficient, economical solutions to practical problems.

Dave Farley, Modern Software Engineering

This definition is one I can live with and marries my experience of creating software to the wider principles of engineering. It also bridges across the two realms of engineering, leaving the differences to practices rather than principles.

It is grounded in practicality rather than aloof theories and it emphasises that capacities drive effective solutions as much as needs. This definition is a huge step forward in being able to build consensus around the purpose of a software engineer.

Standard
Software

In praise of fungible developers

The “fungibility” of developers is a bit of hot topic at the moment. Fungibility means the ability to substitute one thing for another for the same effect; so money is fungible for goods in modern economies.

In software development that means taking a developer in one part of the organisation and substituting them elsewhere and not impacting the productivity of either developer involved in the exchange.

This is linked to the mythical “full-stack” developer by the emergence of different “disciplines” within web software development, usually these are: devops, client-side (browser-based development) and backend development (services).

It is entirely possible for developers to enter one of these niches and spend all their time in it. In fact sub-specialisations in things like responsive CSS and single-page apps (SPA) are opening up.

Now my view has always been that a developer should always aspire to have as broad a knowledge base as possible and to be able to turn their hand to anything. I believe when you don’t really understand what is going on around your foxhole then problems occur. Ultimately we are all pushing electric pulse-waves over wires and chips and it is worth remembering that.

However my working history was pretty badly scarred by the massive wave of Indian outsourcing that happened post the year 2000 and as a consequence the move up the value-chain that all the remaining onshore developers made. Chad Fowler’s book is a pretty good summary of what happened and how people reacted to it.

For people getting specialist pay for niche work, full-stack development doesn’t contain much attraction. Management sees fungibility as a convenient way of pushing paper resources around projects and then blaming developers for not delivering. There are also some well-written defences of specialisation.

In defence of broad skills

But I still believe that we need full-stack developers and if you don’t like that title then let’s call them holistic developers.

Organisations do need fungibility. Organisations without predictable demand or who are experiencing disruption in their business methodology need to be flexible and they need to respond to situations that are unexpected.

You also need to fire drill those situations where people leave, fall ill or have a family crisis. Does the group fall apart or can it readjust and continue to deliver value? In any organisation you never know when you need to change people round at short notice.

Developers with a limited skill set are likely to make mistakes that someone with a broader set of experiences wouldn’t. It is also easier for a generalist developer to acquire specialist knowledge when needed than to broaden a specialist.

Encouraging specialism is the same as creating knowledge silos in your organisation. There are times when this might be acceptable but if you aren’t doing it in a conscious way and accompanying it with a risk assessment then it is dangerous.

Creating holistic developers

Most organisations have an absurd reward structure that massively benefits specialists rather than generalists. You can see that in iOS developer and mobile responsive web CSS salaries. The fact that someone is less capable than their colleagues means they are rewarded more. This is absurd and it needs to end.

Specialists should be treated like contractors and consultants. They have special skills but you should be codifying their knowledge and having them train their generalist colleagues. A specialist should be seen as a short-term investment in an area where you lack institutional memory and knowledge.

All software delivery organisations should practice rotation. Consider it a Chaos Monkey for your human processes.

Rotation puts things like onboarding processes to the test. It also brings new eyes to the solution and software design of the team. If something is simple it should make sense and be simply to newcomer, not someone who has been on the team for months.

Rotation applies within teams too. Don’t give functionality to the person who can deliver it the fastest, give it to the person who would struggle to deliver it. Then force the rest of the team to support that person. Make them see the weaknesses in what they’ve created.

Value generalists and go out of your way to create them.

Standard
Work

Breaking the two-week release cycle

I gave a lightning talk about some of the work I did last year at the Guardian to help break the website out of the two-week release cycle and make it possible to switch to a feature-release based process. It’s the first time I’ve given a public talk about it although I have discussed with friends and obviously within the Guardian as well where we are still talking about how best to adopt this.

I definitely think that feature-releasing is the the only viable basis for effectively software delivery, whether you are doing continuous delivery or not.

In a short talk there’s a lot you have to leave out but the questions in the pub afterwards were actually relatively straight-forward. The only thing I felt I didn’t necessarily get across (despite saying it explicitly) was that this work was done on the big Enterprise Java monolith at the Guardian. We aren’t talking about microapps or our new mobile platform (although they too are released on a feature basis rather than on a cycle) we are talking about the application that is sometimes referred to as the “Monolith”. It was really about changing the world to make it better rather than avoid difficulty and accepting the status quo.

Feature-releasing has real benefits for supporting and maintaining software. On top of this, if you want to achieve collective team effort then focussing on a feature it going to better rather than doing a swath of work in a mini-waterfall “sprint”. The team stands a better chance of building up a release momentum and cadence and from that building up stakeholder confidence and a reputation for responsive delivery.

Standard