Cheat sheet: Retrieval-augmented generation

RAG pipeline:

Ingest: parse → clean → chunk (+overlap) → add metadata → embed → store
Query:  embed query → retrieve top-k → rerank → build prompt w/ context → generate

• Chunk size: small = precise, context-poor. Large = context-rich, noisier. Tune on a labelled set, not by habit.
• Tables: keep headers with rows. Isolated chunks: add a title or summary prefix.
• RAG ≠ fine-tuning. RAG for knowledge, fine-tuning for behaviour. No weight changes.

Hallucination mitigation (RAG reduces, doesn’t eliminate):

1. Explicit “answer only from context, else say I don’t know”
2. Citation per claim (and check the citation is accurate)
3. Reranking before generation
4. Retrieval-quality gate (low top score → refuse). Calibrate the threshold; scores differ across models.
5. Post-hoc verification pass

RAG evaluation: retrieval (precision@k, recall@k, MRR) vs. generation (faithfulness/groundedness, relevance). Use a golden Q&A set with unanswerable questions. Not vibes.

Interview traps: “RAG eliminates hallucination” (false, it reduces it) · treating ANN results as exact · pure vector search without keyword search for exact-term recall.

Quick answers#

• RAG pipeline: ingest (parse→chunk→metadata→embed→index) · query (rewrite→hybrid retrieve+ACL→rerank→prompt→generate+cite→check).
• Locate a failure: gold chunks retrieved? no ⇒ retrieval; yes ⇒ test the generator with the gold passage pasted in.
• Hybrid: BM25 + dense → RRF (score = Σ 1/(k+rank), k = 60 from Cormack et al., SIGIR 2009) → cross-encoder rerank.
• Fix order for bad answers: ingestion → retrieval recall → ranking/precision → context assembly → generation.
• Reasonable defaults to propose (illustrative, verify on your data): hybrid + RRF, rerank a wide shortlist (for example 50) to a few (for example 5), structure-aware chunks with some overlap, metadata/ACL pre-filter, citations + abstain.
• Variants: Advanced (baseline) · Hybrid (exact IDs) · HyDE (phrasing gap) · Parent-child (context) · GraphRAG (multi-hop/global) · CRAG (bad retrieval) · Self-RAG (needs a trained model) · Adaptive (cost) · Agentic (multi-step) · Multimodal.
• Metrics: recall@k, precision@k, MRR (miss = 0), nDCG; faithfulness, answer relevance, correctness, citation accuracy, abstention.

Traps#

RAG “eliminates” hallucination · ACL after retrieval · changing embedding model without re-indexing · judging with 30 queries and no confidence interval · copying a similarity threshold from another model.