Lesson deck
Designing content moderation and guardrail systems flashcards
Reviewing needs JavaScript. You can read every card in the list below.
Loading your cards...
Back to the lesson6 cards in this deck
Read all 6 cards as a list
- Front
- What layers can a moderation system combine?
- Back
- An input classifier, a policy prompt, an output check, a human review lane, and logging with appeals. They are a menu chosen by risk, not one fixed order.
- Front
- Is the moderation precision and recall threshold a technical or business decision?
- Back
- Business: engineering supplies the counts of both error types, and the product owner and policy team choose the point, per category, from the product's risk.
- Front
- Serial, windowed or parallel output moderation?
- Back
- Serial (buffer then release) for severe harm, windowed streaming for mixed risk, parallel with retraction for low severity. Streamed text cannot be un-shown.
- Front
- Why are most blocks false positives when violations are rare?
- Back
- When the false-positive rate is not far below the base rate, the false positives outnumber the true catches. Example: 4,500 caught versus 9,950 honest blocked is 4,500 / 14,450, about 31% precision (illustrative).
- Front
- What is shadow mode in moderation tuning?
- Back
- Run a new threshold or classifier on live traffic, record what it would do, act on nothing, and compare with labels. Sample near the threshold, because random samples of passed traffic hold almost no misses.
- Front
- How do you size the human review lane?
- Back
- Items in the borderline band per day divided by one reviewer's daily capacity, rounded up. 2,000 items at 960 per reviewer per day is 2.08, so 3 reviewers.