AI engineering on AWS
AI that your service
can stand behind.
Generative AI features designed, evaluated and operated with the same discipline as the rest of the service.

Start with the task, not the model
Generative AI can help staff find, summarise and draft, but only if the answers are good enough for the task and someone owns them. A demonstration proves something is possible; operating it needs evidence about quality, failure and cost.
We design, evaluate and operate AI features with the same discipline as the rest of the service: identity, network boundaries, pipelines, monitoring and a named owner for each AI-assisted decision. Sometimes the right answer is search, rules or a clearer form, and we will say so.
We use AI assistants in our own engineering work too. Their output goes through the same review, testing and pipeline evidence as any other change.
When to bring us in
- A promising prototype needs to become something a team can run and defend.
- Staff spend hours finding or summarising information held in approved documents.
- You need to decide whether an AI use case is worth pursuing, and how to tell.
What we take on, and what stays with your team
We take on: the use-case assessment, the engineering of the AI feature, its evaluation suite and guardrails, and the operating procedures.
Your team keeps: the judgement of answer quality, the decision on acceptable error, information governance, and accountability for decisions the AI assists.
Engineering scope
Retrieval over approved sources
Amazon Bedrock for model access and Bedrock Knowledge Bases or a purpose-built retrieval layer. Retrieval respects access controls, so users only see answers drawn from content they may see.
Evaluation baseline
An evaluation set of representative, awkward and adversarial questions with expected answers agreed by the service owner, run on every change and compared with the baseline, including a non-AI alternative where relevant.
Abstention and escalation
The feature says when it does not know, and hands over to a person, rather than guessing.
Guardrails and prompt injection
Bedrock Guardrails for content and sensitive information, limits on what actions the model can trigger, and defences against prompt injection through retrieved documents.
Model change and withdrawal
Model, prompt and source changes go through review and rerun the evaluation. There is a plan for model versions being retired, and for withdrawing the feature without taking the service down.
An illustrative project
Illustrative example, not a client engagement
Starting state. Staff answer recurring questions by searching several hundred policy documents. A prototype chatbot gives fluent but sometimes wrong answers.
Agreed work. Agree the task and acceptable error with the service owner, build an evaluation set from real questions, add retrieval with citations and access control, introduce abstention and guardrails, and compare against the current search.
Deliverables. A use-case decision record, the AI feature with guardrails, the evaluation suite and baseline results, monitoring for quality, cost and misuse, and runbooks for model changes and withdrawal.
Acceptance. Results on representative and adverse cases are compared with the baseline, unresolved limitations are documented, and a named owner accepts operating responsibility.
What we need from you
- A service owner who can judge answer quality, and representative questions with agreed answers.
- Access to the approved content and its access rules.
- Contacts in information governance and security, and a view on whether an Algorithmic Transparency Recording Standard record is needed.
Costs, risks and trade-offs
Model calls, retrieval, evaluation runs and human review all cost money. Measure cost per useful task, not just cost per request, and compare it with doing the task without AI. More capable models cost more and are not always more accurate on a narrow task.
FinOps and cloud cost engineering covers how running costs are made attributable and managed.
Related
Read Putting a generative AI feature into service and Verifying AI-generated code and tests.
See anonymised experience. Compare ways to work together.
A sensible first step
Bring one candidate use case. We will discuss the task, the data involved, the consequences of a wrong answer and how success could be demonstrated.
A first conversation does not commit either side to anything, and we will not ask for access to your systems until a scope is agreed.