Coding, project defense and mock rounds

Concept#

The first lesson in this pair, RAG interview drills and mock, covered the question bank, the 45-minute design mock and the scoring rubric. This lesson covers the other half of interview preparation: coding drills (small functions you may be asked to write), project defense (answering follow-ups about work you actually did) and behavioural stories. It ends with a plan that turns all the drills into three mock rounds. It links back to the rubric and Drills A to C where needed.

Why it matters#

A design answer can be rehearsed. A project defense cannot be faked for long, because each follow-up asks for a detail only the person who did the work would know. A made-up number is the easiest thing to catch: the next question asks how you measured it. Honest answers (“I did not measure that; here is how I would”) are shorter and survive follow-ups.

Coding drills matter for a different reason. A small function such as rank fusion shows whether you understand the idea well enough to write it, not just name it. Writing and checking one by hand is a cheap test of that.

Mental model#

Think of a witness statement. A witness is believed for what they can show and for how clearly they separate what they saw from what they assume. One claim that falls apart under questioning makes the court doubt the rest. A project defense works the same way: label each claim as measured, estimated or unknown, and the follow-ups become easy, because you already know which label applies.

Drill D: coding practice#

Provenance: generated practice.

• Implement chunk_text(text, size, overlap) with edge cases.
• Implement cosine-similarity top-k with NumPy, then explain the approximate alternative.
• Implement reciprocal rank fusion over two ranked lists.
• Compute recall@k, MRR and nDCG@k from ranked results and gold sets.
• Write a minimal RAG loop (embed, retrieve, prompt, generate) with citation ids and an abstain path.

Reciprocal rank fusion merges several ranked lists into one. Each document earns 1 / (k + rank) from every list it appears in, the scores are added, and the documents are sorted by the total. The constant k softens the gap between the top ranks; 60 was fixed in a pilot study (Cormack et al. 2009).

# RRF reference (write yours first, then compare)
from collections import defaultdict

def rrf(ranked_lists, k=60):
    scores = defaultdict(float)
    for lst in ranked_lists:
        for rank, doc in enumerate(lst, start=1):
            scores[doc] += 1.0 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

Check by hand: with lists [A, B, C] and [B, C, D] and k = 60, the scores are A = 1/61 = 0.0164, B = 1/62 + 1/61 = 0.0325, C = 1/63 + 1/62 = 0.0320 and D = 1/63 = 0.0159. The fused order is B, C, A, D. In this example, documents ranked by both lists beat documents ranked by one, and A beats D only because it was first in its list.

That pattern is not a rule. A document ranked first in one list only scores 1/61 = 0.0164, while a document ranked 100th in both lists scores 2/160 = 0.0125, so the single-list document wins. Fusion rewards agreement between lists, but a strong rank in one list can still beat weak agreement. If a document has the same rank r in both lists, it beats a rank-1 document from one list only when r is below 62; at r = 62 they tie, and beyond that the single-list document wins.

Drill E: project-defense prompts#

Provenance: generated practice. Use your real project and real numbers. Never invent a metric to sound better.

Why RAG? What were the sources and the volume? How did you choose chunking, and how did you validate it? Which embedding model, and why? How did you measure quality, and what was in the golden set? What was the biggest failure and how did you fix it? What were the real latency and cost numbers? How did you handle permissions? What would you change at 10 times the scale? What did you try that did not work?

If you have no real number, say so and say how you would measure it. That answer scores higher than a made-up figure that collapses on the first follow-up.

Drill F: behavioural hooks#

Provenance: generated practice. Prepare true stories for: a quality regression you diagnosed, a trade-off between latency and quality, a time you pushed back on using an LLM where a simpler solution worked, and a time you managed expectations about hallucinations.

Mock plan: three rounds#

The drills named here (A to C) are in RAG interview drills and mock; D to F are above.

Run the mock with a partner or the course mentor acting as interviewer only. The interviewer asks one question at a time, picks follow-ups from what you actually said, and does not teach during the mock. Afterwards, score with the rubric in RAG interview drills and mock, then review. A mock is a simulation; it is not a prediction of any real interview.

One-page cheat sheet#

This is the one-page recap for all six drills (A to F), so it sits at the end of the pair.

• Pipeline: parse, chunk, embed, index, rewrite, hybrid retrieve, rerank, assemble, generate, verify.
• Fix order when answers are bad: ingestion, retrieval recall, ranking precision, context assembly, generation.
• Common starting defaults: hybrid with rank fusion, rerank 50 down to 5, structure-aware chunks with overlap, permission filter inside retrieval, citations plus abstain, stream, cache, trace everything.
• Metrics: recall@k, MRR, nDCG, faithfulness, answer relevance, abstention rate, p95 latency, cost per query.
• Say aloud: “I would measure first, then add complexity for a named failure.”

Interview perspective#

Provenance: common industry pattern; no company-specific claim.

Coding and project rounds commonly ask for a small function plus follow-ups about trade-offs, and project discussions commonly probe the numbers you quote. Interviewers differ in how they weigh these, and this course makes no claim about any specific company or role.

Activity#

A 35-minute solo or paired session.

1. Write rrf from memory in 10 minutes, then run it on [A, B, C] and [B, C, D] and compare with the by-hand scores above.
2. Invent two new lists of three documents, compute the fused order by hand, then check it with your function.
3. Pick five questions from Drill E and answer them aloud about a real project. After each answer, label every number you gave as measured, estimated or unknown.
4. For each unknown, write the one measurement that would replace it.
5. Write a four-line version of one Drill F story: the situation, what you did, the result, and what you would do differently.

Homework#

Write a one-page defense sheet for your real project. For each of the ten Drill E questions, write your answer with its evidence, or “not measured” and the measurement you would add. Then schedule Round 3 of the mock plan with a partner, and score it with the rubric.

Quick revision#

• Reciprocal rank fusion adds 1 / (k + rank) per list, then sorts by the total.
• Agreement between lists helps, but a rank-1 in one list can beat a deep rank in both.
• Label each project claim as measured, estimated or unknown.
• If you did not measure it, say so, separate a target from a result, and name the measurement you would add.
• Rounds: questions (A, B), a design you have not seen (C), then defense and coding (D, E).

Check yourself#

• Checkpoint question (answer aloud, no notes): Fuse [M, N, O] and [N, P, M] with k = 60. What is the order, and why is P ahead of O?
• Independent re-test: The interviewer asks for your project’s p95 latency and you never measured it. What do you say in two sentences?
• Common misconception to avoid: Inventing project metrics to sound better, and assuming a document in both lists always outranks one in a single list.

Related#

• RAG interview drills and mock
• Designing a RAG QA system
• Designing an agent product
• Designing content moderation and guardrail systems
• Production deployment of RAG