Generative AI
SearchStart here

Module deck

Safety and guardrails flashcards

Reviewing needs JavaScript. You can read every card in the list below.

Loading your cards...

Back to the module8 cards in this deck

Read all 8 cards as a list

Guardrails, safety and prompt injection

  1. Front
    What is prompt injection?
    Back
    Text that changes what a model does in a way its operator did not intend. It can arrive through the user or through content the model reads.
  2. Front
    Core defence principle against prompt injection?
    Back
    Treat retrieved and tool-returned content as data, never as instructions, and assume some injections will still succeed, so limit what a hijacked model can do.
  3. Front
    Direct vs indirect prompt injection?
    Back
    Direct: the user's own input is the attack. Indirect: an innocent user's agent reads third-party content (web page, email, retrieved document, tool output) that carries hidden instructions.
  4. Front
    False positive vs false negative in a guardrail?
    Back
    False positive: a legitimate request is blocked. False negative: an attack passes. A stricter threshold trades one for the other, so set it by the cost of each failure.

Limiting blast radius and treating output as untrusted

  1. Front
    What is least privilege (privilege separation) for agents?
    Back
    Give each agent and tool only the tools, data and permissions the task needs, and keep read capabilities apart from write or send capabilities, so a successful attack has a small blast radius.
  2. Front
    When should an agent require human approval before acting?
    Back
    Before irreversible, high-value or external-facing actions such as payments, deletions, sending messages and publishing.
  3. Front
    What is the lethal trifecta?
    Back
    An agent that combines access to private data, exposure to untrusted content and the ability to communicate externally is at risk of data theft. Avoid having all three in one agent (Simon Willison, 2025).
  4. Front
    What is improper output handling?
    Back
    Passing model output to a browser, database, shell or other system without validating or encoding it. Prompts can steer output, so treat it as untrusted input (XSS, SQL injection, code execution).