RAG evaluation and hallucination
- Intermediate
- 25 min
- Lesson 5 of 6 in Retrieval-augmented generation
Select text in the lesson first to keep it with your note. With the keyboard, move focus here and press the button: the note goes with the section you are reading.
In this module
On this page
Before you start
- Completed:RAG architecture and chunking
You will be able to
- Explain why RAG reduces but does not eliminate hallucination, including three ways a model can fail with the right context in the prompt.
- Choose among grounding instructions, citations, reranking, retrieval gates and verification passes, and state the limit of each.
- Classify a failure as a retrieval failure or a generation failure from a trace of retrieved chunks, scores and the final prompt.
- Describe how to measure RAG quality with separate retrieval and generation metrics on a golden set, and how much to trust a small improvement.
Merrow holds up the page. Platform 4, in plain print, first line.
“It sounded right,” Tobias murmurs, and announces Platform 7 to a full concourse, warmly and without a pause.
Merrow looks from the page to the intercom. Was the page wrong, or the reading?
Concept
HallucinationHallucinationFluent output that is not supported by the source or by fact. RAG reduces it by grounding answers, but does not eliminate it.Open the glossary entry is when a model states something false or unsupported with confidence. RAG reduces it by giving the model retrieved text to answer from. It does not eliminate it, because the model can still ignore the retrieved context, misread it, or blend it with prior knowledge that the context does not support.
Why it matters
“We added RAG, so it won’t hallucinate anymore” is a common and wrong claim. Interviewers use it to test whether you know RAG mitigates the problem, what to do about the remainder, and how you would measure the result.
In production the cost is concrete. A confident wrong answer about a policy, a price or a platform number gets acted on.
Mental model
RAG hands the model the right textbook page. A student can still misread that page or answer from memory instead.
GroundingGroundingTying an answer to supplied sources, such as retrieved passages, so claims can be checked.Open the glossary entry reduces the opportunity to hallucinate. It does not remove the capability. So you add checks around the model, and you measure.
How a model fails with the right context
Three distinct failures, each with a different fix:
- Ignores the context. It answers from its training data because that felt more familiar. Fix: stronger grounding instruction, less noise, a better model.
- Misreads the context. It takes the right passage and draws the wrong conclusion (wrong platform, wrong date, a negation dropped). Fix: clearer chunk text, extraction steps, verification.
- Blends context with unsupported claims. It adds a plausible detail the passage never stated. Fix: citations per claim and a verification pass.
And one failure that is not the model’s: the retrieved text itself may be wrong, stale or missing. The next sections separate these.
Mitigation techniques
Each technique helps and each has a limit.
- Explicit grounding instruction. “Answer only from the provided context; if it does not contain the answer, say you don’t know.” Limit: it is an instruction, not a guarantee, and models can still drift. Test its wording for loopholes.
- CitationsCitationA pointer from a claim in the answer to its source chunk. Its presence does not prove the source supports the claim, so check citation accuracy.Open the glossary entry or attribution. Require the model to cite which chunk supports each claim. Unsupported claims become visible. Limit: a citation can point at a chunk that does not say what is claimed, so check citation accuracy, not just presence.
- Reranking. Put the best evidence first and remove weak chunks so the model has less to be distracted by. Limit: it cannot fix evidence that was never retrieved.
- Retrieval-quality gate. If the best result’s score is too low, refuse instead of guessing. Limit: similarity scores are not comparable across embedding models or query types, so a fixed threshold copied from elsewhere will misfire. Calibrate it on a labelled set and watch both wrong refusals and wrong answers.
- Post-hoc verification. A second pass checks the answer’s claims against the retrieved context. Limit: it adds latency and cost and the checker can be wrong.
Reading a trace
When an answer is wrong, open the trace for that request. A useful trace records the question, the retrieved chunks with their ids and scores, the final prompt, and the answer.
| What the trace shows | Classification |
|---|---|
| Correct chunk not retrieved, or ranked out | Retrieval failure |
| Correct chunk in the prompt, answer contradicts or ignores it | Generation failure |
| Correct chunk in the prompt, answer follows it, but the chunk itself is wrong or outdated | Source failure |
| No relevant chunk retrieved, yet the model answers confidently | Retrieval failure plus a missing abstention rule |
Without logged chunk ids you are guessing. Log them for every request.
Measuring quality
You cannot judge a RAG system by whether a few answers looked right. Measure retrieval and generation separately so you know what to fix.
- Retrieval metrics: precision@k (how much of the top-k is relevant), recall@k (was the evidence found), MRR (how high the first relevant chunk ranks). RAG evaluation in depth computes them.
- Generation metrics: faithfulnessFaithfulnessWhether an answer's claims are supported by the retrieved context. It is a generation metric, separate from whether the answer is correct.Open the glossary entry or groundedness (does every claim follow from the retrieved context) and answer relevance (does it address the question).
- AbstentionAbstentionA system declining to answer, or asking a clarifying question, when retrieval is empty or weak. Calibrate the gate on labelled data.Open the glossary entry rate: how often it says “I don’t know”, and whether that is right. Include questions the corpus cannot answer.
- Golden setGolden setA curated set of questions with known good answers or sources, used to score a system on every change. Keep slices tagged and add real failures over time.Open the glossary entry: a fixed list of questions with expected answers or gold passages, run on every change. Tools such as RAGAS automate parts of this (Es et al., 2023).
How much to trust a small gain
With a golden set of 30 questions, one question is worth 3.3 percentage points. A 3-point improvement therefore means a single question flipped, which is well within what chance or a prompt change can do. Before acting on it, enlarge the set, check which questions changed and why, and rerun to see how stable the result is.
Real-world application
Production RAG needs an evaluation pipeline the way software needs unit tests: a golden set run against every prompt, retrieval or model change, so fixing one question cannot silently break ten others. See Observability and eval pipelines.
Interview perspective
Provenance: common industry pattern; no company-specific claim.
Expect “your RAG chatbot hallucinated a policy that doesn’t exist, debug and prevent it” and “how do you know the system is good”. Strong answers inspect the trace, classify the failure, name two or more mitigations with their limits, and finish with a metric that would prove the fix. Weak ones claim RAG removes hallucination or rely on “it looked right to me”.
Practice questions
Provenance: generated practice.
- Basic: name two techniques that reduce hallucination in RAG.
- Intermediate: what is the difference between a retrieval metric and a generation metric?
- Advanced: why might a system hallucinate even when retrieval returns the correct chunk?
- Product company: users say the bot “sounds confident but is often wrong”. Design a fix using at least two techniques from this lesson.
Activity
Do, notice, apply.
- Do. Write a system-prompt instruction that forces citation-based answers and an explicit “I don’t know” fallback.
- Notice. Attack your own wording. Find three inputs that would get around it (a question the context half-answers, a context with a negation, a user who asks the model to “just guess”).
- Apply. For each loophole, say which other mitigation from the list would catch it and what it would cost.
Homework
Design a 10-question golden set for a knowledge base of your choosing (real or hypothetical). Include at least two questions the corpus cannot answer and write the expected-answer criteria for each question.
Quick revision
- RAG reduces hallucination. It does not eliminate it.
- Right context can still be ignored, misread or blended with unsupported claims.
- Mitigations: grounding instruction, citations, reranking, retrieval gate, verification. Each has a limit.
- Classify failures from the trace: retrieval, generation or source.
- Evaluate retrieval and generation separately on a golden set. Small gains on small sets are noise.
Check yourself
- Checkpoint (answer aloud, no notes): A RAG bot invents a refund policy. Give a debugging order, three mitigations, and the metric that would prove the fix worked.
- Re-test (nearby variant): Your golden set has 30 queries. How much do you trust a 3-point improvement, and what would you do next?
- Misconception to avoid: “RAG eliminates hallucination”, and evaluating only end to end so a regression cannot be localised.
Related
Practice
Test what you just read. Quizzes and cards are grouped by module: Retrieval-augmented generation.
Check yourself9 questions on what this lesson set out to teach
Checking answers needs JavaScript. You can still read every question and option below.
Keyboard: arrow keys move between options. Space or Enter selects the focused option.