For SRO · SIRO · Service owner
Key points
- Across the four examples examined, the most convincing evidence concerns well-defined tasks: answering questions from published guidance (GOV.UK Chat), extracting planning information from documents (MHCLG's Extract), simplifying documents into Easy Read (Swindon Borough Council) and preparing appeal evidence for officers (Warrnambool City Council).
- Reported gains are significant but measure different things. GOV.UK Chat's 90% figure is an internal accuracy benchmark and its 73% usefulness figure comes from a user survey; none of its measures independently shows that people completed their journey. MHCLG said Extract could complete work that might take an officer up to two hours in about two minutes for some document types, and every output is still reviewed by an officer.
- The evidence comes from different kinds of source, from government pilots and evaluations to supplier-reported case studies, so each figure should be read with its origin in mind.
- What remains unestablished is what happens after the task: whether people get the outcome they need, whether errors are corrected efficiently, whether the service remains trustworthy as policies, data and technology change, what services such as Extract and Swindon's Easy Read work cost to run in full, and whether officers can meaningfully disagree with a prepared recommendation.
- Evaluation needs to extend beyond the model and the immediate productivity measure. A dependable service needs authoritative information, testable behaviour, monitoring and a route to challenge or correct outputs, with assurance proportionate to the severity, reversibility, scale and vulnerability involved.
Artificial intelligence has become a prominent feature of public-sector technology programmes, accompanied by promises of greater productivity, improved access to services and more time for professionals to exercise their judgement. The difficulty for anyone trying to establish what government has achieved is that the evidence emerging from these programmes varies considerably in quality, scope and relationship to the outcomes public services exist to deliver.
A system that summarises a document in seconds may save useful time, although the benefit depends on whether the result is accurate, whether someone must review it and whether corrections create additional work. An assistant that answers questions about government guidance may perform well against a test set without resolving the difficulties faced by someone whose circumstances are unusual or whose records require investigation. An AI casework tool may help officers assemble material more quickly, yet the validity of the eventual decision continues to depend upon the evidence, the applicable rules and the authority of the person making it.
These distinctions deserve attention because an apparent gain in one part of a service can leave the overall experience unchanged, while an error that appears minor in technical reporting may have significant consequences for a resident. The published examples are encouraging, but they are most useful when their claims are examined alongside their limitations, rather than treated as proof of sector-wide transformation.
GOV.UK Chat: measuring the answer and the experience
The Government Digital Service (GDS) has provided unusually detailed accounts of its development and evaluation of GOV.UK Chat, a conversational assistant grounded in published government guidance. In March 2026, GDS reported two public pilots conducted over approximately 18 months, during which more than 10,000 users asked about 26,000 questions. Accuracy on GDS's own benchmark improved from 76% to 90%, while a follow-up survey of GOV.UK app users found that 73% considered the service useful and 64% were satisfied with it. The figures indicate meaningful progress, although an accuracy benchmark, an in-scope answer rate and a user satisfaction survey describe different aspects of performance. None independently establishes that a person completed the service journey they had started. [1]
GDS's May 2026 engineering account is revealing because it describes the work behind the assistant's answers. The team evaluated groundedness, relevance and answer rate, factual accuracy, factual completeness, reliability and reputational safety, drawing on automated tests, expert review and live monitoring. It also introduced metadata filtering and ranking to favour current authoritative content, semantic retrieval using Amazon Titan embeddings, question routing and safeguards operating before and after generation. [2]
There is an important distinction between the March and May accounts. The March pilot findings explicitly describe introducing clarifying questions when a user's original question is ambiguous, while the May engineering article explains question routing and partial answers that steer users towards relevant guidance. These are related features, but the sources describe them differently, and the operational detail should not be flattened into a single generic claim. [1][2]
GOV.UK Chat currently draws on published GOV.UK guidance and does not access a user's personal government records. Consequently, it can help someone understand the published rules without necessarily resolving an individual case, particularly where the answer depends on information held by a department, the interpretation of disputed circumstances or a decision that must be made by an authorised officer. That boundary is a useful feature of the evidence, since it makes clear what the measured system is actually attempting to do. [1]

Extract: converting records into usable planning data
The Ministry of Housing, Communities and Local Government's Extract programme addresses a more specific operational problem. Planning information remains locked in scanned documents, archived maps and historic records whose contents must often be interpreted manually before they can be used in digital systems. Extract identifies text and map features, including some handwritten annotations, and converts them into structured planning information that an officer checks before export. [3]
On 17 June 2026, MHCLG announced that Extract was live and available free of charge to every local planning authority in England. At that point, 50 authorities were actively using it and more than 1,000 documents had been processed during its early rollout. The department said that work which could take an officer up to two hours could be completed by the tool in approximately two minutes for some document types, and that around two-thirds of outputs needed only minor corrections before they were ready for use. Every output still undergoes officer review. [3]
These are concrete measures of deployment and document-processing efficiency, but their scope matters. The figures do not establish the complete cost of maintaining the service, nor do they prove that planning decisions have become faster or more equitable. The quality of downstream outcomes will also depend on how accurately information survives extraction, review, publication and subsequent use, since a boundary incorrectly encoded in a shared dataset may travel further than an error confined to a single paper document.
Extract is therefore an instructive example of how the value of AI can depend upon the less visible engineering and professional work around it, including source provenance, review workflows, correction and the management of data once it leaves the original document.
Accessible information: the value and responsibility of simplification
Swindon Borough Council described a different application in December 2023, using Amazon Bedrock to explore how complex public documents could be converted into Easy Read information. The account, written by a council officer and published on the AWS Public Sector Blog, describes involving people with learning disabilities in decisions about text size, spacing, images and the accessibility of language, which gives the work a stronger grounding in the circumstances of its intended users. [4]
The council reported that manually adapting a typical five-page document could cost approximately £500 and take two weeks, while its AI-assisted approach could produce pages for approximately 7 to 10 pence each. These figures illustrate the potential difference in production cost, but the publication does not provide a comparable total cost that includes quality assurance, correction and sustained operation, nor does it establish how extensively the solution was subsequently deployed. [4]
Accessibility nevertheless represents a form of public value that cannot be reduced to transaction cost. Information may be published and legally available while remaining difficult for some people to understand, leaving them dependent on others to interpret decisions and explain their choices. Simplification creates its own responsibilities because an omitted qualification or altered condition can change the practical meaning of guidance, particularly when the reader has fewer opportunities to detect the mistake.
Casework: faster recommendations and continuing responsibility
Warrnambool City Council in Australia provides a comparison involving case preparation rather than general information access. An AWS customer case study published in 2026 describes an AI-assisted parking appeal process, developed with Symphony3 using Amazon Bedrock and Bedrock AgentCore, that assembles relevant records and prepares a recommendation for an officer who retains the final decision. The council and its partners developed and tested the service over two years, including consultation with the Victorian State Government and relevant oversight bodies, before its launch on 1 July 2026. [5]
AWS reports that average appeal turnaround fell from approximately 75 days to seven days, while officer review time per appeal declined by about 80%. These are significant reported improvements to the process, although they come from the supplier's customer story rather than a published independent evaluation. The case study also notes that appeals remaining undecided after 90 days were automatically cancelled, so speed can benefit residents and preserve revenue for the council, depending on the circumstances of the appeal. [5]
The example raises a question that matters to British public administration: whether officers have the time, information and practical authority to disagree with a prepared recommendation when workload and performance expectations encourage rapid acceptance. A formal human approval step can be important, but it tells us less about effective oversight than the quality of the supporting evidence, the visibility of uncertainty and the organisation's ability to examine the decisions made in practice.
Why consequences must be assessed proportionately
The consequences of an AI failure vary sharply with the service, the decision and the people affected. An incorrect summary of a routine internal meeting may be quickly corrected, whereas inaccurate guidance about a benefit entitlement can contribute to missed support or deadlines, and an incorrectly interpreted clinical record may affect treatment when timely intervention is essential. The same technical description of an error can therefore conceal very different implications for safety, financial security and the possibility of remedy.
Healthcare warrants particular care because physical harm may be difficult or impossible to reverse and because patients may have limited capacity to recognise an error in a clinical recommendation. NHS England's published guidance on AI-enabled ambient scribing recognises the opportunities for reducing documentation burden while identifying governance considerations for healthcare adoption; its review of clinical risk-management standards also emphasises the evolving challenges created by AI and complex interactions between systems. These sources establish why healthcare requires a more demanding assurance conversation, without implying that any of the four implementations examined above constitutes a clinical deployment. [7][8]
Other public services may have less immediate clinical risk while still carrying serious consequences. A mistaken housing referral, repeated misleading advice or an appeal that cannot be meaningfully challenged can generate financial hardship, additional administrative work and a loss of confidence in the institution involved. These effects need to be considered through severity, likelihood, detectability, reversibility, the vulnerability of those affected and the number of people exposed, as well as through the service's capacity to explain and correct what has happened.

Public trust also needs a place in this analysis. A service can appear technically reliable while sending people repeatedly towards the wrong process or requiring them to correct the same misunderstanding, and those experiences can damage confidence even when no single incident appears especially severe. Measuring a reduction in contacts or processing time without examining the reasons behind it may therefore conceal unresolved demand or journeys that users have simply abandoned.
What the evidence leaves unanswered
The National Audit Office's March 2024 report offers a useful institutional perspective, having found that AI was not then widely deployed across the bodies surveyed, although 70% of respondents were piloting or planning use cases. The NAO warned that benefits at scale would depend on wider changes to business processes, skills and the conditions for adoption, including persistent barriers involving legacy systems and data access. Its findings describe an earlier stage of government adoption, but the difficulties they identify remain relevant to interpreting later examples. [6]

Across the four implementations considered here, the most convincing evidence concerns well-defined tasks: answering questions grounded in published guidance, extracting planning information, simplifying documents and preparing evidence for appeal officers. The larger claims about public value require further observation of what happens after the initial task has been performed, including whether people obtain the outcome they need, whether errors are corrected efficiently and whether the service remains trustworthy as policies, data and technology change.
This is where engineering discipline and public-service governance meet. A dependable system needs authoritative information, permission boundaries, testable behaviour, recorded versions of relevant components, operational monitoring and a route through which people can challenge or correct its outputs. For consequential decisions, it also needs a credible account of who retained responsibility, what evidence was available and how an outcome could be reconstructed if it was later questioned.
The published projects show that AI can remove substantial friction from specific activities, and some have reported improvements large enough to justify serious attention. They also demonstrate why the evaluation of public-sector technology must extend beyond the model and beyond the immediate productivity measure, towards the quality, continuity and accountability of the service experienced by the people who depend on it.
Sources and evidence notes
- GDS, 5 things we learned testing GOV.UK Chat, 16 March 2026. Government pilot findings, including accuracy and survey measures.
- GDS, Developing GOV.UK Chat, 15 May 2026. Engineering description and six evaluation criteria.
- MHCLG Digital, Extract is here, 17 June 2026. Department-reported early operational measures.
- Swindon Borough Council officer, Easy Read and Amazon Bedrock, 1 December 2023. Council-authored account hosted by AWS; operational follow-up not established here.
- AWS, Warrnambool City Council customer case study, 2026. Supplier-reported operational results; independent validation not identified.
- National Audit Office, Use of artificial intelligence in government, 15 March 2024. Historical UK-wide institutional assessment of adoption barriers.
- NHS England, Guidance on AI-enabled ambient scribing products, updated 29 July 2026. Healthcare guidance used only to contextualise consequences and assurance.
- NHS England, Review of clinical risk management standards, 29 June 2026. Healthcare safety context; not evidence of performance of the four case studies.
Editorial disclosure: This is independent research and analysis of externally documented implementations. None of the projects described is presented as work undertaken by Inference Public or Inference Clinical. Where results come from supplier-published case studies, their origin and evidential limits are made clear.
All insights