Interview Questions — Intermediate

Round: GenAI/LLM depth (required) · Difficulty: at-level for mid-level AI/GenAI engineer Provenance: each question is labelled in the table. None is verified company-specific. Method: answer aloud first, then compare with the points under Strong and note what you missed.

#	Question	Provenance	Lesson
I1	Why is attention O(n²) and what does that imply for context windows?	Common industry pattern	How LLMs work
I2	Walk through a full RAG pipeline (ingest + query).	Common industry pattern	RAG architecture and chunking
I3	Trade-offs in choosing chunk size?	Common industry pattern	RAG architecture and chunking
I4	Explain HNSW. Is ANN exact?	Common industry pattern	Vector databases
I5	Three techniques to reduce RAG hallucination.	Common industry pattern	RAG evaluation and hallucination
I6	What is tool calling; who executes the tool?	Common industry pattern	Function and tool calling
I7	Explain ReAct.	Common industry pattern	Agentic patterns: ReAct, planning and memory
I8	What is prompt injection; core defence principle?	Common industry pattern	Guardrails, safety and prompt injection
I9	App too slow/expensive — which levers?	Common industry pattern	Cost and latency optimization
I10	Fine-tuning vs RAG — when each, when wrong?	Common industry pattern	Fine-tuning vs RAG vs prompting
I11	Why add hybrid (BM25 + vector) search?	Common industry pattern	Better retrieval
I12	What does a reranker do and what does it cost?	Common industry pattern	Better retrieval

I1. Attention cost and context windows

Strong: score matrix is n×n per head → compute and (naively) memory grow O(n²); doubling the prompt roughly quadruples the attention-score work (projection and MLP cost grow only linearly, so total cost grows by less than 4× at short lengths). Mitigations: FlashAttention (IO-aware, same math, less memory traffic), sliding-window/sparse attention, GQA/MQA, KV-cache, prompt caching, and not stuffing context — retrieve instead. Also quality: long contexts degrade (“lost in the middle”). Follow-up: “Context window is 1M tokens — should you skip RAG?” (cost, latency, attention dilution, permissions, freshness, citation — usually no.) Common mistake: saying the whole model cost is exactly quadratic, or answering ‘use a bigger context window’ with no cost or quality caveat.

I2. Full RAG pipeline

Strong: Ingest: load → clean/parse (tables, PDFs) → chunk (+overlap, keep metadata: source, section, ACL, timestamp) → embed → upsert to vector store (+ keyword index). Query: (rewrite/expand query) → embed → hybrid retrieve top-k with metadata filters → rerank → assemble prompt within token budget → generate with grounding + citations → post-checks → log. Names where each stage can fail. Follow-up: “Which stage do you instrument first?” (retrieval — most RAG failures are retrieval failures.) Common mistake: listing components without saying where each stage fails, or applying access control after retrieval.

I3. Chunk size

Strong: small = precise match but loses context; large = more context but diluted embedding and fewer chunks fit the budget. Overlap prevents boundary loss; structure-aware chunking (headings/paragraphs) beats fixed-size; small-to-big (retrieve small, return parent) combines both. Decide empirically on a labelled query set with recall@k/MRR — not by folklore. Follow-up: “Answer quality dropped after raising chunk size 256→1024. Why and how do you find out?” (embedding averages topics → lower precision; fewer chunks in budget; locate failure in retrieval vs generation, measure each.) Common mistake: quoting a ‘best’ chunk size from folklore instead of measuring on a labelled query set.

I4. HNSW / ANN

Strong: multi-layer proximity graph; search enters at sparse top layer, greedily descends to dense bottom layer. Approximate by design: tune efSearch (recall↔latency) and M/efConstruction (graph quality↔build time/memory). Alternatives: IVF (cluster probe, nprobe), PQ (compression). Memory-hungry; deletes/updates are awkward. Always measure recall vs brute force on your own data. Common mistake: treating ANN as exact, or tuning knobs without measuring recall against brute force on your own data.

I5. Reduce hallucination (RAG)

Strong: grounding instruction + “say I don’t know”; per-claim citations; retrieval-quality gate (low score ⇒ abstain); reranking; post-hoc verification (claim-vs-context check / NLI/LLM judge); lower temperature (more repeatable, not a fix for missing evidence); better chunking/hybrid search. States that it reduces, not eliminates, and how to measure (faithfulness, answer relevance, abstention rate). Common mistake: offering a single fix (usually ‘lower the temperature’) and not saying how you would measure that it worked.

I6. Tool calling

Strong: model emits a structured request (tool name + JSON args matching a schema); your application executes it, returns the result to the model, loop continues. The model never runs code. Hence: validate args, enforce authz in the executor, timeouts/idempotency, treat tool output as untrusted. Common mistake: thinking the model runs the tool, or trusting model-produced arguments for authorisation.

I7. ReAct

Strong: interleave Thought → Action (tool call) → Observation until done; grounds reasoning in tool results and lets the plan adapt. Failure modes: loops, compounding errors, cost/latency per step, prompt-injection through observations. Needs step cap, loop detection, and tracing. Common mistake: describing the loop without step caps, loop detection or tracing.

I8. Prompt injection

Strong: untrusted text (user input, web page, email, retrieved doc) is interpreted as instructions because LLMs have no hard instruction/data boundary. Principle: don’t rely on the prompt for security — least-privilege tools, authz outside the model, human approval for irreversible actions, output filtering, isolate untrusted content, treat model output as untrusted input. Indirect injection (via retrieved/tool content) is the one people forget. Common mistake: believing a stronger system prompt is the main defence, or forgetting indirect injection through retrieved content.

I9. Slow/expensive levers

Strong: (cost) shorter prompts, prompt caching, model routing/cascade, smaller/distilled model, semantic cache, batching (the provider batch API for offline work, continuous batching when self-hosting), output length caps, fewer retrieved chunks. (latency) streaming (perceived latency only), parallel tool calls/independent calls, speculative decoding, smaller model, cache prefill, reduce output tokens. Says measure first (tokens in/out, TTFT, p95) and names the quality cost of each lever. Common mistake: optimising before measuring, or listing streaming as a cost or total-latency fix.

I10. Fine-tuning vs RAG

Strong: see F10, plus: fine-tune when you need consistent style/format/domain behaviour, or to distil a big model into a cheaper one, with ≥ hundreds–thousands of quality examples. Wrong when knowledge changes, you need citations/permissions, or you lack labelled data. Mentions LoRA/PEFT cost advantage and eval before/after. Common mistake: treating fine-tuning as a way to add changing knowledge, or skipping a before-and-after eval.

I11. Hybrid search

Strong: BM25/keyword nails exact terms (IDs, error codes, names); vectors capture paraphrase/semantics. Fuse with RRF or weighted scores; then rerank. Pure vector search can miss exact-match queries without any error. Common mistake: saying vector search always beats keyword search, or fusing raw scores from different scales instead of ranks.

I12. Reranker

Strong: cross-encoder scores (query, chunk) jointly → usually more accurate than bi-encoder similarity but O(k) model calls → retrieve ~50–100 cheaply, rerank to ~5–10. Cost: added latency and money; cap k, batch, cache. Common mistake: reranking the whole corpus instead of a shortlist, or ignoring the added latency.