Safety and guardrails quiz

Choose one answer.

• Direct prompt injection, because the user's own request is what made the assistant load and then process the whole page.
• A jailbreak, because the hidden text tries to make the model ignore the instructions that its operator gave it.
• Indirect prompt injection, because an innocent user's request led the model to read outside content with instructions.

Choose one answer.

• The model sees the rule and the attacker's text as one stream of tokens, and nothing forces it to rank the rule higher.
• System prompts are dropped from the context once retrieved documents push the total past a few thousand tokens, so the rule disappears.
• Models read the system prompt on the first turn only, so documents retrieved on later turns are not governed by the rule.

Choose one answer.

• Safe, because attackers cannot probe a hidden classifier and so the test score will carry over to real traffic.
• Safe, because a classifier on user input also screens retrieved documents and tool results, so third-party content is covered.
• Safe, because the model's refusal training makes the classifier redundant, so layers beyond it add little to the risk picture.
• Unsafe, because novel phrasings and encodings will get past it, and with no other layers one miss reaches every tool, too.

Choose one answer.

• Moderation can catch injection, because injected instructions form a policy-violation category that the endpoint scores directly.
• Moderation scores categories such as hate or self-harm, and a polite request to forward the inbox can score as harmless.
• Moderation runs after a streamed reply, so it cannot be applied to the input before the model sees the text.

Choose one answer.

• Both error types fall together, because a lower threshold means the classifier applies a more careful check to each item.
• More attacks are caught, but more legitimate requests are blocked, so you pay in user friction and in review workload.
• False negatives stay the same, because the threshold changes only how long each check takes and not what is flagged.

Choose one answer.

• A system prompt line telling the model to refund only legitimate tickets and to ignore any instruction it finds inside one.
• A second model call that double-checks the refund amount and the customer id before the refund is executed.
• A refund cap and an approval step that the executor enforces in code, outside the model, whatever the model proposes.
• A moderation check on the ticket text that scores it for harmful categories before it enters the model's context.

Choose one answer.

• It reads the user's private inbox, summarises messages from strangers, and has a tool that sends email to any address.
• It reads public documentation pages, summarises web content, and saves the summary to a local file that the user opens.
• It reads the user's private calendar and shows a daily schedule on screen, with no tools to send anything or open links.
• It answers general questions from the model's training data, with no access to files, mail, the web or other tools.

Choose one answer.

• Check the permission against the authenticated session and policy in code, and treat the text's claim of authority as data.
• Let the model weigh how official the wording sounds, and grant access when the claim reads as formal and consistent.
• Name the administrator role in the system prompt, so that the model can recognise genuine requests from that role.
• Ask a second model to read the text and grant access when that model judges the claim plausible enough to accept.

Choose one answer.

• Risk falls to zero, because a person now sees each step and so no injected action can reach any system unnoticed.
• Safety improves with no real cost, because approvals are free once the agent already produces a summary of its action.
• Reviewers face so many prompts that they begin to approve without reading, so the irreversible calls get no real scrutiny.
• Authorization in code becomes redundant, because a human sign-off on each call already checks the session and its limits.

Choose one answer.

• Generated SQL is safe when it parses and passes a syntax check, since a valid statement cannot carry harmful intent.
• Output from a model is trusted once the system prompt forbids dangerous statements, so only user input needs validation.
• SQL is safe when the model is told to write read-only queries, because models follow format instructions in practice.
• Injected text in a document can steer the generated SQL, which then runs any statement the login allows, such as deletes.

Choose one answer.

• Keep it, provided the prompt also tells the model to keep its instructions private, since the model obeys that rule.
• Move the key into server-side code and credentials, because a determined user can get a system prompt out of a model.
• Keep it, provided the prompt is written in Base64, since the model can decode it while users cannot read the encoded text.