This book was originally published in German at the end of the 80s. It described the result of conducting computer based simulations of situations such as running a town or sub-Saharan country. All the situations were fictional but based on real world scenarios and with a rich simulation model. The book is unable to describe how to succeed but instead focuses on patterns of behaviour that were frequently seen when people experienced failure, and often catastrophic failure, in the simulations.

Misunderstanding complex systems with exponential behaviour

The book offers a succinct and insightful picture of factors that are better understood today but often not in combination and not relating to issues of leadership and management. In no particular order these include the very real problem of differentiating linear and exponential processes. The human mind seems to bias towards linear models and struggles to accurately predict the outcomes of changes in the rate of change itself. Of course this situation is even harder at the start of the processes because the two look the same and therefore if you don’t have a clear understanding of the underlying processes there is no way to predict whether something will be linear or exponential.

Failure to understand exponential growth is one challenge but exponential collapse is even harder for our minds to predict and model. The chapter on predator and prey models were particularly fascinating as often there is massive growth in the population size of the predators before a huge collapse in their numbers. If a metric has been exponential and then becomes linear without a deep understanding of the processes at work you can’t tell whether you have encountered a plateau or precipice.

The book also feels that individual decision makers only rarely can hold a complex model in their minds, participants in the study would sometimes deny the information given to them in the briefing once they developed their own incorrect theories of how the simulation was working.

On the difficulty of being successful

One of the reasons the book can’t draw definite conclusions about what strategies are successful is that because there is no universal system that is successful in all circumstances. For example, generally people who asked more questions after each step in the simulation were more successful than those who didn’t, however at some point all the successful participants asked less questions and acted decisively in ways that advanced their goals. They seemed to better manage their need for information against the need to act and observe and were able to tune the mix of activities in an optimal way.

Experience generally seemed helpful but there is a warning about what the book calls “methodism” which I think might have other names now. What it describes though is the misapplication of prior knowledge or tactics. People look for a few identifying characteristics in the situation that match their experience and then they apply techniques or solutions that have worked for them in the past. In doing so they can ignore information in the current situation that contradicts the likelihood that the previous solution is appropriate.

The book uses “elaboration” as a way to measure whether someone’s proposed solution is based on the situation they are presented with rather than one they have encountered before. Elaborated solutions include principles guiding the attempted solution and potential compromises in executing it as well as mitigations against the failure of the attempted solution.

Essentially people who are more likely to be successful use their previous experience to inform their approach to a new problem but are rigorous in their analysis of the new situation and prepared to adapt previously successful approaches to the new situation.

Unsafety buffers

One very practical takeaway was around the use of buffers in safety procedures. Typically when designing a robust procedure you want to allow for issues in following the procedure or the timing of its execution and so on. This means that most safety procedures tell you to action early at a point when the system is quite far from failure and the capacity of the system is quite high. Ironically this means that if you perform the procedure late or incompletely then quite often it will still work.

The book gives the example of Chernobyl as a place where safety procedures were routinely ignored, abbreviated or circumvented because nothing bad ever happened when they were. If you draw the conclusion that the safety procedures are unnecessary or their buffer values are too high and you can use your own heuristically determined values instead then you start down the path to disaster.

It is important to remember that any conservative safety procedure is conservative to give it the maximum likelihood of working in a range of circumstances. One that has a narrow range of applicability is less likely to result in a safe outcome.

As the book points out it is impossible for individual humans to learn from catastrophic failures. Collectively though we should be studying and drawing conclusions from the worst outcomes that we have not personally experienced.

Defining success and avoiding failure

One key takeaway I took away from the book is that while it talks about failure and success even the successful outcomes involved trial and error and contained points where things were not as good as they could have been. Most of the outcomes described as successful involved the participant having an idea of some new stable situation that improved aspects of the current one and working methodically towards it. This is quite a modest definition of success compared to the way it is commonly used in business for example.

The terrifying thing about the book is that in most of the simulations the virtual people involved would probably have been better off if nothing had been done. The scenario usually starts in a stable situation that is sub-optimal and on my reading it seems the majority of participants took that situation and turned it into a hellscape of unsustainable growth or development followed by disaster and a collapse of society to levels below the starting point.

In many ways the book is a justification of small ‘c’ conservatism, sustainable improvements are hard to achieve and the advantage of time-tested solutions is that have been validated under real-world conditions. The counter-argument though is that improvements are possible and to not seek them out of fear is also an unhappy situation.

This is a small book and you can read the essence of it’s content in this paper. Like all the best book its ideas have an impact out of proportion with the amount of time it takes to explain them.

I think I first found about this book via a post from Tim Harford who was buying links if you’re interested (or details to order from your local bookshop).