Officers interpreting permitting rules, housing staff applying an allocations policy, contact centre advisers answering questions about council tax and members of the public trying to understand what applies to them all depend on guidance that is spread across intranets, document stores and publishing systems, often in several versions. An AI assistant can make that guidance far easier to use, but only if it answers from the right version, respects who may see what and can show where every answer came from.
What better looks like, and how you would measure it
Improvement should be measured against how the work is done today. The measures that matter most are: - how often answers cite the correct, current source - how often the assistant declines to answer when the guidance does not cover the question - the time it takes staff to reach a usable answer - the effort needed to check answers before they are relied on - how quickly a change to the guidance reaches the assistant.
If you cannot measure the current position, establishing it is the first piece of work.
Start with the information you already hold
Some of the most consequential failures begin in the information collection and its retrieval controls, even when the model itself performs as designed. Model behaviour, integration and operating arrangements add risks of their own, but before choosing any technology, someone needs to know who owns each document, when it took effect, what has replaced it, who is allowed to read it and how an urgent withdrawal would reach the assistant. Where those answers are unclear, improving the information will do more good than any assistant. Our free AI-ready data checklist sets out 18 questions to ask of a collection before a pilot.
The options, in plain terms
There is more than one sensible way to meet this need, and the right one depends on your information, your users and the consequences of a wrong answer.
- Improve the information first, where ownership, currency or permissions need fixing before any assistant could be trusted.
- Improve conventional search, where people mainly need to find the right document and a generated answer would add risk without adding value.
- Use a workplace product you already have or can readily adopt, such as AWS Quick or Microsoft 365 Copilot, where staff need general help across content those products can reach.
- Use a knowledge base service, such as Amazon Bedrock Knowledge Bases, where a service needs answers from a defined collection. AWS offers two models with different responsibilities. With a customer-managed Bedrock Knowledge Base, your organisation selects and manages the underlying vector store, configures data sources and synchronisation, and remains responsible for application-level authorisation, while Bedrock performs the configured ingestion and retrieval processes. Bedrock Managed Knowledge Base takes on more of the storage, ingestion and retrieval infrastructure, although your organisation must still govern its sources, identities, permissions and use of the results. Our article on RAG as a data product explains the difference in more depth.
- Build a custom retrieval application, where permissions, integration, evaluation or the user experience need more control than managed options provide.
- Use a simpler design without retrieval, where a small, stable body of guidance can be supplied to the model whole, as Defra's public assistant does with its forty-page AI toolkit.
Our comparison of building retrieval, using a managed knowledge base or adopting AWS Quick sets out these trade-offs in more detail.
Who does what
| What AWS provides | What we do | What you retain |
|---|---|---|
| Amazon Bedrock models, available in the London region, with processing locations that vary by model and change over time; Bedrock Knowledge Bases in two forms: customer-managed, where you select and manage the vector store and configure sources and synchronisation while Bedrock performs ingestion and retrieval, and Bedrock Managed Knowledge Base, which takes on more of the storage, ingestion and retrieval infrastructure; Bedrock Guardrails for filtering questions and answers | Assess the readiness of your collection; design ownership, freshness and withdrawal; choose and build the retrieval approach, or decide that none is needed; write evaluation questions from your own policy; test for hostile content hidden in documents; integrate with your systems; prepare the runbook for handover | Ownership of the guidance and its versions, accountability for what the assistant tells people, decisions about public use, transparency records where they apply, and the decision to proceed or stop |
We check model availability and processing locations for each design on the day it is made, because both change.
When answers have consequences
An assistant that only helps staff find a policy carries different risks from one that tells a resident whether they are entitled to something. Errors about entitlement, safeguarding or enforcement can cause real harm even when a person makes the final decision, so the design should set out who checks answers, how mistakes are reported and corrected, and when the assistant must decline. Documents can also carry hidden instructions that try to steer the assistant, so the collection itself needs testing as well as the questions people ask.
In central government, departments and in-scope arm's-length bodies must publish an Algorithmic Transparency Recording Standard record for tools that interact directly with the public or significantly influence decisions with public effect, once those tools reach beta, pilot or production. Whether an internal assistant falls in scope depends on how it is used, and purely internal tools are generally outside the requirement. Local authorities, the NHS and the wider public sector are encouraged rather than required to publish records. Public-facing assistants must also meet accessibility requirements.
What others have published
Each summary reflects the published source as we read it on 20 November 2026.
- Defra runs a public assistant over its AI guidance toolkit on Amazon Bedrock in London. Its transparency record (updated 17 November 2026) describes a public beta and reports that its tests met four of six pass marks, including the two it missed. Read Defra's record
- The Environment Agency is piloting a guidance assistant for around 150 permitting staff on Microsoft Azure. Its record (updated 10 September 2026) describes a pre-deployment pilot with no evaluation results published, and shows the same job being tackled on a different platform. Read the Environment Agency's record
- GOV.UK Chat answers questions from GOV.UK guidance in the GOV.UK app, using Amazon Bedrock. GDS has described how it evaluated the service in its account of developing GOV.UK Chat, and our analysis of published public-sector AI projects sets out what that evaluation does and does not show.
None of these is Inference Public's work.
When we would recommend something else
We would advise against a new AI assistant where the guidance is contradictory or out of date and nobody owns its correction, where better search would meet the need, where your existing licences already cover it, or where the consequences of a wrong answer cannot yet be managed. In each case the assessment says so, and explains what would need to change.
For councils
Councils often hold guidance in many places after years of restructuring, and local government reorganisation will bring separate policies together. Transparency records are recommended rather than mandatory for local authorities, but publishing one helps residents and members understand how the assistant is used. Professional oversight matters as much as the technology: the Department for Education and the Department of Health and Social Care's guidance on AI in social care, for example, expects practitioners to treat AI output as a first draft and keep their own professional judgement.
For central government
Departments and in-scope arm's-length bodies need a transparency record for assistants that interact directly with the public or significantly influence decisions with public effect, once they reach beta, pilot or production; whether an internal assistant is in scope depends on its use, and purely internal tools are generally outside the requirement. Machinery of government changes can also move ownership of guidance and accounts, which our article on machinery of government changes and your AWS Organization explains.
For agencies and regulators
Regulators depend on officers applying complex guidance consistently, and an assistant that surfaces the wrong version can lead to inconsistent decisions affecting regulated parties. Version authority, exceptions and access restrictions need settling before any design, and the assessment examines whether a simpler method would be enough.
A sensible first step
Our AI-ready data assessment examines one collection and its intended use, and ends with an impartial Readiness and Delivery Decision, which may recommend improving the information, using an existing product, managed retrieval, a custom application, conventional search or stopping.
