Interview Questions — Advanced

Round: GenAI/LLM depth + design follow-ups (required) · Difficulty: at-level → stretch for mid-level; at-level for senior Provenance: each question is labelled below. None is verified company-specific. What often separates candidates here (our judgment): strong answers tend to cover evaluation methodology (retrieval vs generation), hallucination control, latency/cost trade-offs, and knowing when not to use an LLM. How to use: answer aloud for 2 to 3 minutes, then compare with the points under Strong.

A1. RAG hallucinates even though retrieval returned the right chunk — why?

Strong: model ignores/under-weights context (parametric memory wins), misreads or over-generalises it, context is long and the evidence sits mid-prompt, conflicting chunks, chunk is relevant but doesn’t actually contain the answer (partial), instruction doesn’t enforce grounding, high temperature (this makes output less repeatable, but is rarely the root cause). Diagnosis: compare faithfulness to retrieved context vs answer correctness; ablate (shuffle/limit chunks, lower temperature, require quotes). Fixes: quote-then-answer, citations, abstain option, rerank to put evidence first, verification pass. Common mistake: blaming the model without checking whether the evidence was in the final prompt, or fixing it only by lowering temperature.

A2. Detect and prevent a repetitive tool-call loop

Strong: hard step/budget caps; detect repeated (tool, args) hashes or no-progress states; progress checks (did state change?); escalate/abort with a useful message; backoff and different-strategy prompt on 2nd repeat; idempotent tools; trace every step. Root-cause the loop (ambiguous tool errors, tool returning empty, missing “done” criterion). Common mistake: adding a step cap and stopping there, with no repeated-call detection or definition of done.

A3. Retrieval-quality gate — what signal says “don’t answer”?

Strong: top-1 score and score gap (margin) vs a threshold calibrated on labelled data per embedding model; number of chunks above threshold; reranker score; query-doc lexical overlap; a lightweight “is the context sufficient?” classifier. Trade-off: false abstentions (annoying) vs confident wrong answers (costly) — choose by domain risk; monitor abstention rate. Raw cosine scores are not comparable across models/corpora. Common mistake: using one raw cosine threshold across models and corpora instead of calibrating on labelled data.

A4. Why a system prompt alone can’t stop prompt injection

Strong: instructions and data share one channel; the model can follow the most persuasive or recent text; adversarial strings are unbounded and attackers adapt; indirect injection via tools/RAG bypasses the user turn. Defence-in-depth: privilege separation, authz in code, allow-listed tools/egress, human-in-the-loop for irreversible actions, content isolation/spotlighting, output validation, monitoring/red-teaming. Common mistake: saying ‘write a stricter system prompt’ as the primary defence.

A5. Semantic caching without wrong hits

Strong: cache key = normalised query embedding + scope (tenant, ACL, model/prompt version, locale, date-sensitivity); high similarity threshold tuned on labelled near-duplicates; verify with a cheap reranker/entailment check; skip cache for personalised, time-sensitive, or tool-using flows; TTL + invalidation on source updates; measure false-hit rate, not just hit rate. Do not share cache entries across permission boundaries. Common mistake: reporting hit rate only, without measuring false hits, or sharing cache entries across tenants.

A6. Plan-then-execute vs ReAct

Strong: plan-then-execute — usually fewer LLM calls, parallelisable, auditable plan; brittle when the world differs from the plan (needs replanning). ReAct — adaptive per observation; more calls/latency, can drift or loop. Hybrid: plan, execute with checkpoints, replan on failure. Pick by environment predictability and cost of mistakes. Common mistake: declaring one pattern always better instead of choosing by how predictable the environment is.

A7. Eval pipeline that isolates retrieval vs generation regressions

Strong: golden set with query → relevant doc/chunk IDs → reference answer/criteria. Retrieval stage: recall@k, MRR/nDCG against labelled IDs (no LLM involved). Generation stage: feed gold context to the generator to measure generator quality in isolation; then end-to-end with real retrieval. Faithfulness/relevance via rubric or LLM-judge calibrated against human labels; track per-slice; gate deploys on thresholds; sample production traffic into the set. Warn about judge bias and small-set noise (CIs). Common mistake: running only end-to-end evals, so a retrieval regression and a generation regression look the same.

A8. Why fine-tuning doesn’t fix hallucination (mechanistically)

Strong: hallucination = fluent sampling from a distribution when the model lacks/mis-weights knowledge; SFT teaches style/format and can even increase confident fabrication when training on facts the base model doesn’t know. Better levers: grounding (RAG), abstention training/eval, calibrated refusal, preference tuning on faithfulness, tool use. Fine-tuning for new facts is unreliable and stale on arrival. Common mistake: saying fine-tuning ‘teaches the model the facts’ without noting it can increase confident fabrication.

A9. Vector-only vs hybrid search

Strong: decide with data: build a query set that includes exact-ID/jargon queries and paraphrase queries; compare recall@k for BM25, dense, hybrid (RRF). Corpora with codes, SKUs, legal citations, error strings → hybrid is almost mandatory; purely conversational/semantic corpora may be fine with dense + reranker. Mention index/ops cost. Common mistake: choosing hybrid or dense by habit instead of comparing recall@k on a query set with exact-ID and paraphrase queries.

A10. Moderation: serial vs parallel with generation

Strong: serial (input check → generate → output check) = safest, adds latency. Parallel = hides latency, but you may generate (and pay for) content you then discard, and streaming risks showing unsafe tokens before the verdict. Common pattern: fast input classifier gates; stream output through a lightweight incremental filter; heavyweight check async for logging/escalation. Tie choice to harm severity. Common mistake: treating serial versus parallel as a latency question only, and ignoring wasted cost and unsafe streamed tokens.

A11. When not to use an LLM

Strong: deterministic rules/regex/SQL suffice; need exactness (arithmetic, compliance calculation); strict latency/cost ceilings; high-stakes with no verification path; small labelled dataset where a classical classifier beats prompting. Uses LLMs where language understanding/generation is the bottleneck and puts deterministic code around them. Common mistake: answering ‘never use an LLM for X’ as a rule, instead of tying the choice to exactness, latency, cost and verification.

A12. Choosing LLM-as-judge safely

Strong: rubric with anchors, pairwise comparison to reduce scale drift, position/length/self-preference bias controls (swap order, normalise length), calibrate against a human-labelled sample (report agreement), use a different/stronger judge model, keep a human audit lane, and do not judge on the same data you used to tune the prompt. Common mistake: trusting the judge without checking agreement against human labels, or judging with the same data used to tune the prompt.