Cheat sheet: Safety and guardrails

Prompt injection:

• Instructions hidden in content the model reads. Direct = the user is the attacker. Indirect = a web page, email, document or tool result carries the attack. A jailbreak is a direct attack aimed at the model’s own safety rules.
• The model has no hard boundary between data and instructions. Treat retrieved and tool content as data, and assume some injections succeed.
• No guardrail fully prevents injection. OWASP says it is unclear whether fool-proof prevention exists (genai.owasp.org, accessed 2026-10-09).

Layers (defence in depth):

• Hardened system prompt (helps, not sufficient) → content marked as data → input screen → moderation → output checks → least privilege, authorization in code, human approval → monitoring and red-teaming.
• Every detector trades false positives, false negatives, latency and cost. Set strictness by the cost of the failure.
• Moderation detects harmful content, not injection.

Limit the blast radius:

• Excessive agency has three causes: too many tools, too much permission, too much autonomy.
• Lethal trifecta: private data + untrusted content + outbound channel. Do not combine all three in one agent.
• Model output is untrusted input: validate, encode, parameterise, sandbox. Keep secrets out of the system prompt.

Interview traps: “the system prompt stops injection” · “a filter solves it” · “moderation catches injection” · giving an agent tools “just in case” · trusting a model-supplied user id · ignoring indirect injection.

Quick answers#

• Injection defence: least privilege + authorization in code + human approval + marked data + output validation + monitoring (not just a system prompt).
• Irreversible actions: confirm with the exact action shown, limit, audit. For idempotency keys and compensation see Function and tool calling.
• For MCP-specific risks see Model Context Protocol.