An in-app AI agent that never claims an action it didn't take

What it is

An in-app AI agent that takes real actions for your users, built to scale with your user base: it holds no state, and each conversation runs in its own isolated session on a managed runtime. Our reference build is the coach inside a fitness app, tested against an automated suite of real conversations: users ask in plain words and it finds routes, books sessions, adds friends and organizes group outings through a few dozen tools over the app’s own API. Replies can open a screen in the app, and it remembers what matters about each user between conversations.

The problem it solves

Language models are good at conversation and unreliable at keeping their word. Asked to book something, a model will sometimes say “done” when the call failed or never happened, cite a record that does not exist, or fill in a detail the user never gave. In a chatbot that only answers questions, that is a wrong answer. In one that changes data, it is a broken promise: a session that was never booked, a friend request that was never sent, and a support ticket when the user finds out.

The usual fixes are more rules in the prompt or a second model grading the first. Both are still guesses. Teams ship a convincing demo, then stall because nobody can say how often the agent lies about what it did.

What you get

• Actions, not just answers: the agent books, creates, edits and cancels through your API, with the same permissions and validation your screens use.
• Replies that match what happened: if a change failed or never ran, the user is told so, and offered a retry.
• No invented details: records, places and dates come from your data or the user’s own words; when something is missing, the agent asks.
• Memory that carries over: each conversation keeps its context, and lasting facts such as an injury or a preferred training time follow the user into new chats.
• Replies that do something: the answer can open a card or a screen in your app, not only show text.
• A safety net that grows: every bug we find becomes a check and a test conversation that keeps passing.

How we deliver it

We start with an AI & Data Sprint: pick the handful of things your users most want to do by talking, describe those operations of your API as specs, wire them as tools, and write the checks for what must never go wrong. We build the test conversations alongside, against your real API and a test account. From there, Systems Integration brings more of your API in as tools, and an Engineering Retainer keeps the checks and tests growing as real users find new edge cases. We need your API, its documentation, and someone who knows which mistakes would hurt most.

Is it for you

It fits when your app already has an API, your users repeat multi-step tasks, and a false “done” would cost you trust or support time. It is not for you if you only need answers over your documentation: a search-and-answer assistant is simpler and cheaper. It is also not the quickest path if you need many languages from day one, because the checks read the agent’s replies and are written per language.

Design goals

1. The agent never tells a user something was done unless a tool result says it was.
2. Records, places and dates come from tool results or the user’s own words, never from the model’s guess.
3. What the model must not choose (where the user is, what day it is, which tools they may use) is set by code.
4. The agent holds no state: any container can answer any message.
5. Every bug found becomes a rule, a unit test or a test conversation that keeps passing.

Architecture

1. People and inputs

   UsersAsk in plain words

   Mobile appSign-in, location, cards

2. Backend

   Chat APIAuth, clock, context

   MemoryHistory and lasting facts

3. Agent

   Agent graphModel, tools, checks, retry

   Language modelManaged, called per step

4. Tools and data

   Tool gatewayManaged, built from API specs

   App APICatalogue and per-user data

   App dataRoutes, plans, friends

Four pieces, and only one of them talks to the model. The chat API verifies the user’s sign-in, loads the conversation’s history and the user’s lasting facts, adds the user’s location and the current time from its own clock, and invokes the agent. The app never holds a model key.

The agent runs in a managed container runtime that isolates each session. It is a small Python service with a health check and an invocation endpoint, so it runs the same on a laptop and in the cloud.

The tool gateway turns the app’s API into tools. Each group of operations is described as an API spec, and the gateway exposes them over the Model Context Protocol, signs the agent’s calls, and injects the right credential per group from its own vault. Read-only catalogue tools carry no user identity; user tools are a separate group with a separate key.

Memory lives in the backend’s key-value table, next to the app’s own data: one item per conversation turn and one per lasting fact.

How the hard parts work

The graph: model, tools, check, retry

1. Model picks a stepCall a tool, or answer
2. Run the toolsResults trimmed, errors kept
3. Check the answerRules in code, no second model
4. Correct and retryForced to act when needed

A false success never ships. If a change failed or never ran, plain code replaces the reply with an honest one, whatever the model wrote.

Each message is a small state graph. The model either calls tools, which run and loop back to it, or gives its answer. The answer itself is a tool call with a schema (the reply, an optional app action, and a few follow-up suggestions), so nothing is parsed back out of free text, and the fields are kept flat because flat schemas are the ones every model and gateway combination fills in reliably. The answer then goes to a check node. If it fails, the graph adds a correction the user never sees and sends it back to the model. Tool turns and retries are both capped, so a confused turn ends instead of spinning.

Deterministic checks around a non-deterministic model

The checks are plain code that compare the reply with what the tools actually did this turn. A write tool whose last call errored must be reported as a failure. A reply that claims a completed action (“I booked”, “I deleted”) with no successful write behind it fails. So does a booking request that ends without a booking and without a question asking for the missing date. Any record the reply points the app to must have come from a tool result this turn. When the fix means doing something rather than rewording, the retry withdraws the answer option and forces a tool call, so the model cannot just apologize again in prose. After the graph ends, the same predicates run once more as a backstop: if the model ignored every correction, code overwrites the reply. Failures our own guards return on purpose, like “ask the user first”, are exempt, so they become questions instead of apologies.

Tools from API specs, with arguments the model never sets

The agent loads the gateway’s tools on every message and wraps each one. The user’s location is stripped from the schema the model sees and merged in by code, so it never asks for a city or guesses coordinates. Responses are trimmed to what the model needs, since the app’s cards render the full detail but errors pass through untouched so the checks see them. Route-creating tools carry a guard: every distinguishing word of each waypoint must appear in what the user wrote this conversation, otherwise the tool refuses and the agent proposes the point instead. User tools are left out entirely when nobody is signed in. A contract test in CI checks that every tool spec still matches a real backend route.

Memory without state in the agent

The agent remembers nothing between calls. The backend sends the conversation’s prior turns on every message and saves the new turn afterwards, which also powers the app’s history screen. Lasting facts are a separate store: the agent saves one with a dedicated tool when the user mentions an injury, a routine or a goal, and the backend sends the user’s facts with every message, in any conversation. The current date is spelled out with its weekday rather than as a bare timestamp, because models are unreliable at working out the day of the week, and bookings are written in local time.

Decisions

Question	Options	We chose	Why
The agent loop	Hand-written loop; framework agent; explicit graph	Explicit graph with a check node	Every path, cap and retry is visible and testable
How the answer comes back	Free text with tags; a schema-validated final tool call	Final tool call with flat fields	Nothing to parse; flat schemas fill reliably
Checking answers	Second model as judge; rules in code	Rules in code, plus a backstop	Same verdict every time, free, unit-tested
How tools are built	Hand-written functions; generated from API specs behind a gateway	API specs behind a managed gateway	One source of truth, credentials kept out of the agent
Where memory lives	In the agent’s process; backend store sent per call	Backend store, stateless agent	The runtime can recycle any container; history is needed anyway
Retrieving lasting facts	Embeddings and ranking; the full list	The full list on every call	A user’s facts stay few; ranking can be added without changing storage

Guardrails

• Security. Users sign in through a managed identity service and the backend verifies every token. The agent reaches tools only through a gateway that checks signed requests, and each tool group’s credential stays in the gateway’s vault, never in the prompt or the app.
• Least privilege. Signed-out users never see user tools. Catalogue tools carry no identity at all. The agent’s runtime can call only its own gateway, and per-user rate limits on lookups and invites stop the agent being used to enumerate users or spam them.
• Cost. Caps on tool turns and retries per message, a small fast model, checks that run in code rather than as extra model calls, and tool results trimmed before they go back to the model.
• Reliability. Rule checks, a code backstop, unit tests for every rule, and a suite of scripted conversations against the real model, gateway and API that resets its test user before each scenario.
• User data. The model sees trimmed results: names, not avatars or private chats. Lasting facts are only what the user said plainly, and location always comes from the device, never from the model.

Failure modes

What breaks	What happens
A write tool errors	The reply says it failed and offers a retry, never “done”
The model claims an action it never took	Sent back and forced to act; if it still does not, code replaces the reply
The model picks a place the user never named	The tool refuses; the agent proposes the place and asks
The model cites a record it never fetched	Sent back; only records from this turn’s tool results are allowed
The model keeps calling tools	Tool turns are capped and it must answer
The agent crashes mid-turn	The user gets a short “try again” line; nothing half-done is claimed

Trade-offs, and when not to use this

The checks are keyword rules over the reply in one language: they catch the failure shapes we have seen, not every possible one, and a new language means writing them again. That is why each new bug becomes a rule and a test conversation. User tools take the user’s id as an argument the agent fills in from its context, because the gateway’s shared credential proves “this is the agent” but not “for which user”; the backend only accepts it with the agent’s key, but for sensitive data the next step is a short-lived token scoped to one user (planned). Replies arrive whole rather than streamed word by word, a retry costs another model call, and a new gateway session is opened per message. If your assistant only answers questions and never changes data, this layer is more than you need.

What we would measure

• Share of turns that pass the checks on the first try.
• Retries and backstop overrides, by rule.
• Pass rate of the test conversations across repeated runs.
• Tool error rate, by tool.
• Reply latency (p95) and tool turns per message.
• Model cost per conversation.