Multi-agent systems: when to use them and what they cost

Story

Hide story

A lamp in car five has been flickering all evening. Wick swaps the bulb and writes DONE in his notebook.

Ten minutes later the night guard, working from his own list, swaps it again, and then the fuse, which was fine. He writes DONE in his notebook. Neither notebook mentions the other.

The lamp works. Who was supposed to be doing that job?

Concept#

• Workflow: the model and tools are orchestrated through predefined code paths. Predictable and cheap.
• Agent: the model directs its own process and tool use. Flexible and less predictable.
• Multi-agentMulti-agent systemSeveral coordinating agents. Justified for broad, parallel work or separate permissions; otherwise it adds tokens, latency and failure modes.Open the glossary entryClose: several agents, each with its own prompt, tools and context, coordinate on one task.

These are Anthropic’s workflow and agent definitions (Building effective agents, 2024-12-19). “Multi-agent” is a common extension of them. The guiding principle is to find the simplest solution that meets the quality bar and add complexity only when it demonstrably improves outcomes.

This lesson is about choosing the structure and what it costs. The next lesson, Agent evaluation, is about measuring whether the structure you chose works.

Why It Matters#

“Design an agent” questions test judgment about when not to build a multi-agent system, and how you would bound the one you build. Every extra agent adds tokens, hand-offs and ways to fail, so you need a reason for each one.

Mental Model#

A workflow is an assembly line: every station and its order are fixed. A single agent is one capable generalist who decides what to do next. A multi-agent system is a small team with a manager. The team can work on several things at once, but it spends time on meetings, and a misunderstanding between two members can ruin the result.

Technical Deep Dive#

Choosing the structure#

Start with a single model call plus retrieval and examples. Move to a workflow when the steps are known. Move to an agent when the number of steps cannot be predicted. Move to several agents only when one agent’s limits (context, tools, parallelism) are the real bottleneck.

The patterns#

Anthropic’s guidance names five workflow patterns:

• Prompt chaining: a fixed sequence of calls, with optional checks between steps.
• Routing: classify the input, then send it to a specialised handler.
• Parallelization: run independent subtasks at the same time (sectioning), or run the same task several times and combine the answers (voting).
• Orchestrator-workers: a lead model decomposes the task at run time, delegates to workers and combines their results.
• Evaluator-optimizer: one call generates, another critiques, and the loop repeats until the criteria are met.

A supervisor with hand-offs (one agent assigns work, or passes control to another) is a common variation of orchestrator-workers. Names differ between frameworks, so describe the shape instead of relying on a label.

When multi-agent helps, and what it costs#

It helps when the work is broad and parallel (researching many sources), when separate windows keep each worker’s context clean, or when different parts need different tools and permissions. Anthropic’s research-system write-up describes subagents that work in parallel with their own context windows and pass condensed findings to a lead agent (Anthropic, 2025-06-13).

It costs a lot. The same write-up reports that agents typically use about four times the tokens of a chat interaction and multi-agent systems about fifteen times, so the task has to be valuable enough to pay for it. It also says domains where all agents must share one context, or whose steps depend closely on each other, are a poor fit today, and that most coding tasks have fewer parallelisable parts than research. Those are one team’s observations from 2025, not laws.

Other costs: added latency from coordination, duplicated work, harder debugging, errors that compound, and a larger security surface. A multi-agent design needs a shared-state plan, clear task contracts between agents, termination rules and a budget.

Reliability math: errors compound#

If each step succeeds with probability p and steps are independent, an n-step chain succeeds with probability p to the power nError compoundingIf each step succeeds with probability p, independently, n steps all succeed with probability p to the power n, so long chains fail more than single steps suggest.Open the glossary entryClose. Illustrative numbers: 0.95 per step over 10 steps gives 0.95^10, about 0.60. Real steps are not fully independent, so treat this as a warning about the direction, not a forecast. It is why shorter chains, verification steps, checkpoints, retries and replacing free-form steps with a fixed workflow matter, and why adding agents (more steps and hand-offs) can lower reliability.

Diagram#

Diagram: A lead agent splits the task across three workers that each have their own context and tools, then combines their results.

Real-World Application#

A deep-research feature: an orchestrator starts parallel searchers, each returns a short sourced brief, a citation checker validates the claims, and per-run token budgets cap the cost.

Interview Perspective#

Expect “single versus multi-agent” and “the agent loops and the bill explodes”. A strong answer starts with the simplest design and justifies any extra agent by parallelism, context isolation or tool separation. It names the cost, uses the p to the power n argument for long chains, and mentions budgets and termination rules. Common mistakes: choosing multi-agent by default and ignoring cost.

Practice Questions#

These are generated practice questions, not reported from any company.

• Basic: What is the difference between a workflow and an agent?
• Intermediate: When does orchestrator-workers beat a single agent, and what does it cost?
• Advanced: Why can adding agents lower the end-to-end success rate of a task?
• Product-Company: Design a deep-research feature. Would you use one agent or several, and what would you cap?

Activity#

For a customer-refund use case, sketch it three ways: (a) a workflow, (b) a single agent, (c) a multi-agent design. For each, list the likely steps, the main failure mode, and the rough relative cost. Pick one and defend it in three sentences. If you picked (c), name the exact reason a single agent was not enough.

Quick Revision#

• Simplest structure that meets the bar. Patterns: chain, route, parallelise, orchestrator-workers, evaluator-optimizer.
• Multi-agent buys parallelism and isolation at a large token and coordination cost.
• Errors compound: p^n over n steps.
• Shared state and clear task contracts help prevent agents from redoing each other’s work.

Check yourself#

• Checkpoint question: A 6-step chain has 0.9 success per step. What is the end-to-end rate (about 0.53), and what would you change?
• Independent re-test: Convince me a multi-agent design is worth it for one specific task, and say what it will cost.
• Common misconception to avoid: “More agents means a smarter system.”

Related#

• Agent evaluation: state, trajectory, pass^k and release gates
• Agentic patterns: ReAct, planning and memory
• Designing an agent product
• Context engineering and memory