Function and tool calling

Story

Hide story

Wick has a key ring with three keys on it: the luggage hold, the dispatch safe and the ticket stamp. A note on the counter says, in a hurried hand, “Sort out tonight’s late parcels.”

“Right away,” says Wick, and his hand is already on the ring. The note does not say which key.

Which keys does he get tonight, and who hands them over?

Concept#

A language model produces text. It cannot query your database, send an email or read a clock. Tool callingFunction calling (tool calling)The model emits a structured request with a tool name and arguments, and your code runs it and returns the result. The model never executes anything itself.Open the glossary entryClose (also called function calling) lets the model answer with a structured request instead of prose: a tool name plus arguments, usually JSON that matches a schema you supplied. Your application decides whether to run it, runs it, and puts the result back into the conversation.

Why It Matters#

Tool calling is the primitive under every agent. The agent loop is this round trip repeated. It is also where most production risk sits, because a model-written request is about to touch a real system. Interviewers use it to check that you know the model only proposes and your code decides.

Mental Model#

The model does not call the function. It writes a note: “I would like to call get_weather(city='Delhi').” Your code reads the note, decides whether to honour it, runs the function, and hands the result back as a new message. The model then continues from that result.

Think of the model as a very capable but untrusted colleague who can only fill in forms. You still own the filing cabinet.

Technical Deep Dive#

The round trip#

1. You send the user message plus a list of tool definitionsTool definitionThe name, description and parameter schema that tell the model what a tool does. It is all the model knows about the tool, so vague ones lead to wrong calls.Open the glossary entryClose.
2. The model either answers in text or returns one or more tool calls (name and arguments).
3. Your code validates and executes each call.
4. You send the results back, tied to the call they answer, and the model continues. It may answer, or request more calls.

The exact message shapes differ by provider. In Anthropic’s API a call arrives as a tool_use block (the response has stop_reason: "tool_use") and you reply with a tool_result block. In OpenAI’s API you append a function_call_output item that carries the original call_id. The idea is the same: a call id links each result to its request. (Checked against both vendors’ docs on 2026-10-09; API details change, so read the current docs before coding.)

Your tools and hosted tools#

Where the code runs is the main thing that differs between tools.

• Client tools are the ones you define. Your application executes them. This lesson is mostly about these.
• Provider-hosted toolsHosted toolA tool such as web search or code execution that runs on the provider's infrastructure. Less code for you, less control, and possible extra charges.Open the glossary entryClose (web search and code execution are examples) run on the provider’s infrastructure. You see the results without writing a handler. Anthropic calls these server tools and OpenAI calls them built-in tools.

So “the model never executes anything” is the right default, and the precise version is “the model emits text; some system outside the model executes it, and for your own tools that system is you.” Hosted tools trade control for convenience: less code to write, but you inherit the provider’s behaviour, limits and extra usage charges.

What the model chooses among#

The model picks only from the tools you list. It decides when to call (by default, on each turn it may call a tool or answer directly) and what arguments to pass. Most APIs let you steer this with a tool_choice setting: let the model decide, require some call, force a specific tool, or allow none. Parallel calls (several in one turn) are supported by both vendors’ APIs and can be switched off.

Designing a tool definition#

A tool definition is the only thing the model knows about your function. Treat it like documentation for a new teammate who cannot ask questions.

• Name: short, specific, verb-first (create_invoice, not invoice). Two similar tools with vague names make the model guess.
• Description: say what the tool does, when to use it and when not to. Anthropic’s guidance on agents compares this to writing a good docstring for a junior developer.
• Parameters: a JSON schema with types, enums where the values are a closed set, and a short description per field. Mark truly required fields as required.
• Make mistakes hard: Anthropic reports that requiring absolute file paths removed a class of errors in a coding agent. Prefer one unambiguous argument shape over several clever ones.
• Few, well-separated tools: every definition costs input tokens on every request, and many overlapping tools can make selection harder (Anthropic documents a tool search tool for very large tool sets; see Model Context Protocol for the 100-tool case).

Some APIs offer a strict modeStrict modeA tool-calling setting that makes arguments follow the schema more reliably. It does not check permissions or whether the values make sense.Open the glossary entryClose that makes the model’s arguments follow your schema exactly (strict: true in both Anthropic’s and OpenAI’s APIs). Strict mode improves format reliability. It does not make a value correct, allowed or safe, so it does not replace the executorExecutorThe application code that receives a model-proposed tool call and runs it. It must validate arguments, authorize from the real session, apply timeouts and treat results as untrusted.Open the glossary entryClose checks below. Keep your own schema validation as defence in depth even with strict mode on.

What your executor must do#

The arguments are model output, and the model may be steered by a user, a web page or a document. Treat them as untrusted input, like a field in a public web form.

Why not in the prompt? A prompt is a request, not an access control. A model can ignore it, be talked out of it, or follow a competing instruction hidden in retrieved text. Rules that must always hold belong in code that the model cannot rewrite.

Cost of these controls: more code and some added latency per call. The alternative is an agent whose worst day is a double refund.

Diagram#

Diagram: The model produces a tool call as text, your application validates and executes it against the real system, and the result goes back to the...

Real-World Application#

Search, calculators, database queries, sending email and running code are all built on this round trip. A common production shape is a thin executor layer between the model and every tool, which validates, checks the caller’s permissions, enforces timeouts, logs each call, and returns structured errors.

Interview Perspective#

Expect “walk me through what happens when a model calls a function” and “how do you handle a tool call that fails or times out”. Common mistakes are saying the model runs code, trusting model-supplied ids for authorization, and forgetting idempotency when retrying a side-effecting call. Good answers also mention logging each call and bounding the number of calls.

Practice Questions#

These are generated practice questions, not reported from any company.

• Basic: What does the model actually output when it “calls a tool”?
• Intermediate: Why define a schema for each tool instead of letting the model call anything freely?
• Advanced: A tool call times out after it may already have charged a card. What should happen next?
• Product-Company: Design the retry and error-handling behaviour for a tool that calls a flaky third-party API.

Activity#

Write a tool definition for a hypothetical send_email(to, subject, body) tool: name, description (including when not to use it), JSON schema, and which fields are required. Then list three things your executor must check before it actually sends, and decide which of them could be abused if the model were being steered by text in an incoming email.

Homework#

Sketch the full message sequence (user, model, tool call, tool result, model, final answer) for the task “what’s a 15% tip on a $42 bill, and email me the answer”. Mark where the executor validates, where it needs an idempotency key, and what it returns if the email service times out.

Quick Revision#

• Tool calling: the model emits a structured request; something outside the model executes it.
• Round trip: call, execute, result linked to the call id, model continues.
• Your tools run in your code; provider-hosted tools run on the provider.
• The definition is the model’s only knowledge of the tool, so write it carefully.
• Validate, authorize from the real session, time out, retry with bounds, use idempotency keys, treat results as untrusted.

Check yourself#

• Checkpoint question: Who executes a tool call, and what must the executor validate before running it? Give two security and two reliability controls.
• Independent re-test: The model passes a user_id that is not the caller’s. Where should this be prevented, and why not in the prompt?
• Common misconception to avoid: Thinking the model “runs” tools, or trusting model-produced arguments for authorization.

Related#

• Agentic patterns: ReAct, planning and memory
• Model Context Protocol and tool ecosystems
• Guardrails, safety and prompt injection