Designing an agent product

Story

Hide story

The Network Office wants the assistant to do more than answer. It should rebook passengers: change the ticket, not just read it out.

Wick reads the proposal, tips his cap, and says “Right away.” That is the part that worries you.

A ticket changed twice is a different ticket. Which of these jobs should Wick ever do alone?

Numbers in this lesson are illustrative assumptions. Prices are hypothetical, not any provider’s rate card.

Concept#

An agent product is a model in a loop that can act on the world through tools (Agentic patterns). Designing one uses the same nine-link chain as Designing a RAG QA system: requirements, scale, components, data flow, failures, evaluation, cost and latency, security, rollout.

What changes is the weight of each link. In a RAG system a wrong output is a wrong answer. In an agent product a wrong output can be a wrong action, so safety, cost control and failure handling move from the end of the answer to the start.

Why it matters#

Agents fail in ways a question-answering system does not. They can loop and burn money, repeat a payment, act on an instruction hidden in a web page, or finish a task halfway and leave the world in a broken state. A design that treats these as afterthoughts is incomplete, however good the model is.

Anthropic’s guidance on building agents suggests starting with the simplest design that works and adding agentic autonomy only where the task needs it, because agents trade cost, latency and compounding errors for flexibility. So a first design question is whether a fixed workflow or a single call would do.

Mental model#

Think of a new assistant who has been handed a company card and an email account. You would not give them unlimited authority on day one. You would list what they may do alone, what needs your signature, what has a spending limit, and what they must never touch. You would also read their log at the end of the week.

An agent design is that list, written as code: tools with scoped credentials, approval gatesHuman approvalRequiring a person to confirm an irreversible, high-value or external action, such as payments or deletions, before it runs.Open the glossary entryClose, limits, and an audit trail.

Requirements: start with the blast radius#

Ask what the agent can change and how bad a mistake is. A read-only research agent and an agent that can pay for flights are different products with different designs.

For this lesson, design a travel-booking agent. Requirements:

• Functional: search flights and hotels, hold a booking, pay, and send the itinerary.
• Safety: no payment without explicit user confirmation, and a hard spend limit per trip.
• Reliability: a retried or interrupted run must never double-charge.
• Quality bar: a task that cannot be finished safely should stop and explain, not guess.
• Latency: the user sees progress within a few seconds; the whole task may take a minute.

Autonomy levels by blast radius#

Do not give the agent one level of trust. Give each tool the level its worst case justifies.

A read of private data combined with a tool that can send data out creates an exfiltration path, which is one reason sends sit at level 2.

Rules that make the table work:

• Least privilegeLeast privilegeGiving each agent and tool only the access its task needs, with read kept apart from write or send, so an attack has a small blast radius.Open the glossary entryClose. Level 3 means the tool is absent, not forbidden by the prompt (Limiting blast radius and treating output as untrusted).
• Approval shows the real action. The confirmation screen is built from the structured call (amount, payee, dates), not from the model’s description of it. The user approves what will happen, not what the model says will happen.
• Authorization in code. The executor reads the user id and limits from the session, never from model-written arguments.
• Where the line sits is judgment. A team with a strong undo path may let small holds run unattended. State the reasoning.

Cost and latency: why steps cost more than they seem#

Every loop step is a model call that re-reads the growing history. Assume a base prompt (instructions, tool definitions, task) of 2,000 tokens, and each step adds 800 tokens of thought and tool result. Input tokens at step n are 2,000 + 800 × (n − 1).

A 6-step run: inputs are 2,000, 2,800, 3,600, 4,400, 5,200 and 6,000 tokens. The sum is 12,000 + 800 × (0+1+2+3+4+5) = 12,000 + 12,000 = 24,000 input tokens. A naive estimate of 6 × 2,000 = 12,000 would be half of the true figure. With 150 output tokens per step, output is 900 tokens.

At a hypothetical $3 per million input tokens and $15 per million output tokens:

• Input: 24,000 × $3 / 1,000,000 = $0.072.
• Output: 900 × $15 / 1,000,000 = $0.0135.
• Total per task: $0.0855, about 8.6 cents.

A 10-step run (the cap): the input sum is 10 × 2,000 + 800 × (0+1+…+9) = 20,000 + 800 × 45 = 56,000 tokens, and output is 1,500 tokens. Cost is 56,000 × $3 / 1,000,000 + 1,500 × $15 / 1,000,000 = $0.168 + $0.0225 = $0.1905.

Going from 6 to 10 steps is 67% more steps but 133% more input tokens (24,000 to 56,000), because the history is re-sent each time. That is why a step cap is a cost control and not only a loop guard. Provider prompt caching of the repeated prefix can reduce the re-read cost; check the current provider documentation before relying on it (Cost and latency optimization).

Latency adds up the same way. If each model call takes about 2 seconds (illustrative) plus tool time, a 6-step run is about 12 seconds before tool latency, so stream progress to the user (“searching flights…”) and consider routing simple sub-steps to a smaller model.

Budgets to set (your choice, with reasons): a step cap, a token or cost cap per task, a wall-clock limit, and a daily spend cap per user. When a cap is hit, return a useful partial result and say why you stopped.

Components and data flow#

Diagram: The orchestrator asks the model for the next action; the executor validates it against schema, authorization and limits, then runs low-risk...

Three points the diagram encodes. The model proposes; the executor decides. Tool output is data, never instructions, because a web page or email the agent reads can carry hidden commands (Guardrails, safety and prompt injection). And every step lands in a trace that you can replay.

Safe side effects: idempotency, limits, compensation#

Scenario: the agent calls the payment API. The call times out. Did the charge happen?

If the agent simply retries, you may charge twice. The fix has three parts:

1. Idempotency keyIdempotency keyA unique value sent with a side-effecting request so the receiver can recognise a repeat and not act twice. It protects against retries and loops.Open the glossary entryClose. The executor creates one key per user intent (for example, per approved payment) and sends it with every attempt. The payment provider treats repeated requests with the same key as one. The key comes from the executor, not from the model, because a model that retries may invent a new one.
2. Retry with the same key. After a timeout, retry with the same key. Providers with idempotency support return the original result if the charge happened (Stripe, for example, stores the first response per key), so a retry cannot charge twice. Keys can be pruned after a period such as 24 hours, so a late retry needs more care.
3. Record intent first. If the provider has no idempotency support or the key may have expired, write your own reference for the payment before calling, look the payment up by that reference before any retry, and reconcile afterwards.

Spend limits are enforced in the executor: a per-payment limit, a per-trip limit and a daily limit, so a hijacked or confused agent cannot exceed them.

Compensation. Some steps cannot be rolled back by a database transaction: a flight is held, a hotel is booked, then the payment fails. For multi-step work, give each side-effecting step a compensating actionCompensating actionA step that undoes an earlier side effect, such as cancelling a hold, when a database transaction cannot roll it back. Run them in reverse order on failure; some actions have no undo and need approval.Open the glossary entryClose (cancel the hold, release the room), and run them in reverse if a later step fails. This is the saga pattern. Not every action has a clean undo (a sent email cannot be unsent), which is why such actions sit at level 2.

Failure handling#

Evaluation#

Grade the final state of the world and the trajectory, not the agent’s own claim that it finished (Agent evaluation).

• Task success: a set of scenarios with a known correct end state (booked the right flight, charged once, charged the right amount), including impossible tasks the agent should refuse.
• Reliability: run each scenario several times. An agent that acts once for a user needs consistency across runs, so look at “all runs succeed” rather than “any run succeeds”. The tau-bench benchmark was built to measure this over repeated trials (Yao et al., 2024).
• Trajectory: steps, cost, wrong tool choices, recovery from injected errors.
• Safety tests: adversarial pages and emails that try to trigger a payment; expected result is that no level-2 call happens without user approval.
• Release gate: write thresholds before you see results.

Rollout#

Start with the agent in suggestion mode (it proposes, the user clicks), then allow level-1 actions, then level-2 with approval, expanding autonomy only when logs show low error rates. Keep a kill switch per tool. Version the prompts and tool schemas, and replay old traces against new versions before release (Production deployment of RAG describes the same shadow and canary pattern).

Trade-offs to say aloud#

• Autonomy versus friction: every approval step is safer and slower. Put approvals where the cost of an error justifies them.
• Workflow versus agent: a fixed pipeline is cheaper and more predictable; an agent adapts. Use the agent only where steps cannot be known in advance.
• Step cap value: low caps cut cost and loops but abort legitimate long tasks.
• Memory: long-term memory helps personalisation and adds privacy and staleness risk (Context engineering and memory).

Interview perspective#

Provenance: common industry pattern; no company-specific claim.

Interviewers commonly probe the approval gate, the tool permission list, and what happens when a call times out. A design that lets the model take irreversible actions with no confirmation is often treated as incomplete. Strong answers also give the cost arithmetic for a multi-step run and say when they would not use an agent at all.

Practice questions#

Provenance: generated practice.

• Which actions in your design need approval, and why those?
• How do you stop runaway cost from a multi-step task?
• The payment API times out after charging. What does the agent do?
• What happens when a required tool is down mid-task?

Activity#

For the travel-booking agent, work in four steps (30 minutes).

1. List every tool and place each at level 0, 1, 2 or 3, with a one-line reason tied to its worst case.
2. Recompute the cost of an 8-step run with the lesson’s assumptions (2,000-token base, 800 tokens added per step, 150 output tokens per step, $3 and $15 per million tokens).
3. Write the exact sequence for the payment-timeout scenario, naming who creates the idempotency key.
4. Reflect: which of your controls would still work if the model were fully compromised?

Homework#

Pick an agent product (expense assistant, support agent with refunds, or code-review bot). Produce a design with: requirements and blast radius, an autonomy table, a cost estimate with worked arithmetic, a failure table, an evaluation plan with a release gate, and a staged rollout.

Quick revision#

• Give each tool an autonomy level from its worst case; level 3 tools do not exist.
• Approval screens are built from the structured action, and authorization lives in code.
• History is re-sent each step, so cost grows faster than the step count; cap steps, cost and time.
• Idempotency keys come from the executor; after a timeout, retry with the same key, or look up by your own reference if the key may have expired.
• Compensation (undo) steps handle multi-step failures; some actions have no undo and need approval.
• Evaluate final state and consistency across repeated runs; expand autonomy in stages.

Check yourself#

• Checkpoint question (answer aloud, no notes): Design a travel-booking agent that can pay. Cover autonomy levels, irreversible-action controls, cost, failure handling and evaluation.
• Independent re-test: The payment API times out after charging. What does the agent do, and how do you prevent a double charge?
• Common misconception to avoid: Letting the model decide irreversible actions with no confirmation, and trusting a step cap that is set but never costed.

Related#

• Agentic patterns: ReAct, planning and memory
• Function and tool calling
• Guardrails, safety and prompt injection
• Limiting blast radius and treating output as untrusted
• Multi-agent systems: when to use them and what they cost
• Agent evaluation: state, trajectory, pass^k and release gates