The talks from this session are now online.

Transcription is popular with social workers (as with most people!) and studies in pilots demonstrated productivity improvements in terms of the time taken to create a care report or to document an interactions. This is one area that is deemed an AI success by most of local and central government so it was interesting to hear about a study into the aftermath of adoption (something that is much less investigated).

It seems that time savings were sustained but different councils took advantage of that in different ways some to reduce the amount of work people were doing others to increase the throughput of cases the social workers were meant to complete.

There was a good question from the audience on whether the quality of the reports was measured before the transcription was introduced. A key dimension of measuring the quality of the new workflow is that it is relative to the previous output, overall the result may be bad but if it is still better or as good as the previous output the process has actually been improved.

I get frustrated in my own work with people just cutting and pasting auto-generated meeting transcripts and notes but often the people posting them would have got them wrong before or, even more likely, would have not done any. Despite the complaints of workslop it can be argued that correcting a bad summary causes less problems than the absence of the summary in the first place.

In social care if the majority of the transcript is correct and on record then perhaps that overall that is better for people using care systems as it is closer to their actual needs and situation.

The other interesting finding is that specialised transcription tools seemed to do better than generalist tools. Providing the appropriate context makes a difference and I’m really not sure what reason there is to use a general tool for these kinds of purposes beyond perhaps cost.

I think that we’re lacking iteration strategies in AI adoption currently. Too many trials feel like they aim to generate a “win”, some process gets adopted and then everyone moves on to the next thing. Generalised tools have been deemed “good enough” in the trial and possibly that means there is now a barrier to removing a sub-optimal tool despite the impact on those using services.

The final takeaway I had is that accents are still poorly handled and there doesn’t seem to be any significant pressure on providers to improve the situation. This really should be a space where government could bring pressure for substantial change.

It is was interesting to see a detailed evaluation of these types of tools essentially after adoption. Since they have already been touted as a success by the adopters I don’t think there is a lot of incentive to look deeply into the results with both the users of the tools and the people who are affected by the quality of their output but that needs to change.

The other talk was also very insightful but I don’t really have a meaningful commentary on except that the burden of fixing broken systems is very much on the system users and it was the common feeling of anger and disappointment to see that people who have experienced being in the care system once again being expected to work to fix not only the record of their care but also the accountability systems within government.

The TransformGov website has more information about previous and upcoming talks