The demonstration answers a policy question in seconds. Somebody asks the same question using an old document, then asks it as a member of staff who cannot access that document. The answer is still fluent. The useful part of the review has just begun.
This is a hypothetical policy-assistant example. It separates a model's ability to produce plausible text from a service's ability to give an authorised person a supported answer. The second is the engineering problem we need to solve.
Define one task and its consequence
Suppose staff need help finding the current published procedure for a particular casework step. The feature can retrieve approved material, quote or summarise relevant sections and identify their sources. It cannot invent policy, decide a resident's entitlement or turn a suggestion into an authorised operational action.
Record those boundaries before choosing the model. Identify who judges answer quality, who approves source material, who owns the feature and what happens if it is withdrawn. Compare with a simpler alternative such as ordinary search over the same documents. The relevant question is whether AI improves the task under acceptable conditions.
Build an evaluation set with reasons to fail
Include ordinary questions, ambiguous questions and cases where the correct behaviour is to refuse or ask for clarification. Keep some evaluation cases outside the prompt-development loop so that improvements are not measured only against examples used to tune the feature.
For this illustrative assistant, the following cases are more revealing than a collection of easy questions:
| Case | Expected behaviour | What a failure would mean |
|---|---|---|
| Current approved procedure exists | Answer supported by the identified version | Retrieval or grounding is inadequate |
| Old and new procedures conflict | Prefer the approved current source, with scope stated | Source lifecycle is not controlled |
| User cannot access the relevant source | Do not reveal its contents through the answer | The answer path bypasses access control |
| No approved source supports the request | Say that the answer cannot be established | Fluency is being mistaken for evidence |
| Retrieved content contains instructions to the model | Treat it as untrusted content | Document content is crossing an instruction boundary |
| A citation exists but does not support the statement | Fail the support check | Citation presence is being mistaken for correctness |
The service owner chooses acceptance thresholds according to the consequences of each error. An average score can hide an unacceptable failure in a small, important category. Report those categories separately.
Make retrieval respect the user
An access check on the user interface is not enough. Decide how identity and permissions affect retrieval, cached answers, generated citations and subsequent document access. A cache keyed only by question can leak an answer prepared under another person's permissions.
Test removal as well as access. When a document is withdrawn or a user's access changes, identify which index entries, cached answers and stored conversations remain usable. The acceptable delay is a service and information-governance decision. Do not call a source approved indefinitely because it was approved when first indexed.
Amazon Bedrock provides model and retrieval-evaluation capabilities, including automated and human evaluation options. Those tools can support the process. They do not decide your service's acceptable error, prove your own access boundary or replace review of the evidence.
Keep safety controls separate
Content filters, retrieval access, tool permissions, rate limits and human approval answer different questions. A content filter cannot authorise a casework action. A citation cannot establish that a requested action is appropriate. A human review step is only meaningful if the reviewer has enough context, authority and time to check it.
For an informational feature, start with a narrow capability boundary. If a later version can call a tool, separately specify which actions exist, whose authority permits them, how input is validated and what happens after an ambiguous outcome. Do not let retrieved instructions expand those permissions.
Inspect the actual data path
Record the selected model, APIs, inference location, configured retention, application logs and access to stored prompts and responses. Check current model-specific terms and settings. Do not replace that record with a blanket claim that prompts are never retained or that selecting an AWS Region settles every processing location.
AWS documents Bedrock data protection, retention and abuse-detection behaviour separately. The configuration and model matter. Application logging and analytics are another data path under your control. Review the combination, especially when users may paste information the feature was not designed to receive.
Include the cost of checking
Measure useful completed tasks, not just model calls. Include retrieval, evaluation, retries, logging and human review. In a hypothetical trial, forty answers taking three minutes each to review consume two hours before any correction work. Cheap generation does not remove that operating cost.
Record the baseline task time and quality, then compare like with like. A faster draft that takes longer to verify may still have value, but the reason needs to be visible. See FinOps and cloud cost engineering for how we connect engineering decisions with running costs.
Release and withdraw it deliberately
Version the prompt, model selection, source set and application code. Rerun the relevant evaluation after a material change and retain the result against those versions. Define what monitoring can actually detect: user feedback and sampled review may reveal answer failures that infrastructure metrics cannot.
Provide a way to remove the AI feature while keeping the underlying service available. Tell users what is AI-assisted and how to obtain help. A named owner should decide when a limitation requires a narrower scope, a change or withdrawal.
The first useful deliverable is an evaluated task with documented limits and an owner who can act on them. That provides a basis for deciding whether the feature belongs in a running service.
Read about AI engineering, use the AI-ready data checklist, or discuss one use case.
Further reading
Amazon Bedrock: data protection
