Top 100 Priority Questions (GenAI Engineer)

Rounds: GenAI/LLM depth · GenAI system design · Resume/Projects (required) · Coding and Behavioral are in their own banks. Naming note: the page URL still says top-100-most-asked so old links keep working. The list is a set of priority questions, not a count of what companies ask. Provenance: every item carries a label in the Provenance column: Common industry pattern (topics that recur across GenAI engineering interviews) or Generated practice. Common industry pattern means the topic recurs across interview guides and the field; the wording and ordering are ours. Generated practice means we wrote the prompt as practice. None is verified company-specific. The priority stars are a judgment about importance, not measured data. Check a company’s current process before relying on any of this. Priority (our judgment, not data): ★★★ high priority · ★★ common · ★ differentiator for senior or strong candidates. How to use: cover the answer, answer aloud in 60 to 90 seconds, then compare. Retry the ones you missed with a nearby variant. Deep dives are linked per section.

Each table row: question, answer key, one follow-up, provenance. Where a row overlaps a question in a per-level bank, the answer key points to it; those banks also list a Common mistake for each question: Fundamentals, Intermediate, Advanced, Scenario Based.

A. LLM fundamentals (How LLMs work, Tokenization and embeddings)

#	Priority	Question	Answer key	Follow-up	Provenance
1	★★★	What does an LLM compute at each step?	Distribution over next token (logits→softmax), a decoding rule picks one, append, repeat.	Where does randomness come from? See F1 in Fundamentals.	Common industry pattern
2	★★★	Explain self-attention (Q/K/V).	Scores = QKᵀ/√d, softmax weights, weighted sum of V; causal mask; multi-head; O(n²).	Why √d? Why positional info? See F2 in Fundamentals.	Common industry pattern
3	★★★	What is a KV-cache?	Cache past K/V so decode only computes the new token; memory grows with length × layers × KV heads × batch.	Prefill vs decode? See F4 in Fundamentals.	Common industry pattern
4	★★★	Why transformers over RNNs?	Parallel training, direct long-range paths, scale; cost O(n²).	Where do SSM/RNN-style models still win? See F3 in Fundamentals.	Common industry pattern
5	★★★	What is tokenization and why not words?	Subword (BPE etc.) balances vocab size and coverage, handles unseen words; cost/context vary by language.	Why do numbers/code tokenize oddly? See F5 in Fundamentals.	Common industry pattern
6	★★	Temperature, top-k, top-p?	Temperature reshapes distribution; top-k/top-p truncate the tail; use low for extraction, higher for creativity.	Why can temperature 0 still vary between runs? See F6 in Fundamentals.	Common industry pattern
7	★★	Context window limits and “lost in the middle”?	Cost and attention dilution; evidence mid-prompt used less reliably; order and trim.	Mitigations?	Common industry pattern
8	★★	Encoder vs decoder vs encoder-decoder?	Decoder-only dominates generation; encoders for embeddings/classification.	Where does an embedding model fit?	Common industry pattern
9	★★	What is an embedding and why cosine similarity?	Dense vector capturing meaning; cosine compares direction, robust to magnitude.	When prefer dot product? See F8 in Fundamentals.	Common industry pattern
10	★★	Pretraining vs SFT vs RLHF/DPO?	Knowledge → format/behavior → preference alignment.	What does each fix?	Common industry pattern
11	★★	What is MoE (mixture of experts)?	Router sends each token to a few experts, so total parameters grow while per-token compute tracks only the active experts. Costs: all experts must sit in memory, plus load balancing. Not covered as a lesson in this course.	Serving implications?	Common industry pattern
12	★	GQA/MQA?	GQA shares K/V heads across groups of query heads (MQA uses one K/V head): smaller KV-cache, faster decode. GQA usually costs little quality; MQA usually costs more.	Effect on concurrency?	Common industry pattern
13	★	Why do LLMs hallucinate?	Trained to produce plausible continuations, not to verify; knowledge gaps; no calibrated abstention.	What reduces it?	Common industry pattern

B. Prompting and structured output (Prompting fundamentals, Reasoning models and structured outputs)

#	Priority	Question	Answer key	Follow-up	Provenance
14	★★★	Zero-shot vs few-shot vs CoT?	Examples steer format/behavior; CoT helps multi-step reasoning at token cost.	When does CoT hurt? See F7 in Fundamentals.	Common industry pattern
15	★★★	How do you get reliable JSON?	Schema-constrained decoding/tool calling, validate, bounded retry, fallback.	Does valid mean correct?	Common industry pattern
16	★★	How do you version and test prompts?	Treat as code: version, eval set, regression gate, canary.	What if the provider updates the model?	Common industry pattern
17	★★	System vs user prompt roles?	Priority and intent separation, but not a security boundary.	Why not?	Common industry pattern
18	★★	When to use a reasoning model?	Hard multi-step tasks, high cost of error, latency tolerance; not for extraction/FAQ.	How measure the gain?	Common industry pattern
19	★★	Self-consistency?	Sample N, majority vote; ×N cost.	Alternatives?	Common industry pattern
20	★	Prompt caching?	Static prefix first; changes early invalidate later cache.	Prompt layout?	Common industry pattern
21	★	Few-shot example selection?	Dynamic retrieval of similar examples, diversity, edge cases.	Risks (leakage, bias)?	Common industry pattern

C. Embeddings, vector search, chunking (Embeddings and similarity search, Vector databases, RAG architecture and chunking)

#	Priority	Question	Answer key	Follow-up	Provenance
22	★★★	How do you choose chunk size?	Small = precise/low context; large = context/noisy; tune on eval with overlap and structure-aware splits.	What if answers span chunks? See I3 in Intermediate.	Common industry pattern
23	★★★	How does HNSW work?	Layered proximity graph, greedy descent; recall/latency/memory knobs (M, efSearch).	Compare IVF/PQ. See I4 in Intermediate.	Common industry pattern
24	★★★	Dense vs sparse vs hybrid?	Semantics vs exact tokens; fuse with RRF.	Why RRF over score sums? See I11 in Intermediate.	Common industry pattern
25	★★★	What is reranking and why?	Cross-encoder scores query+chunk jointly; retrieve 50, rerank to 5.	Latency cost? See I12 in Intermediate.	Common industry pattern
26	★★	Choosing an embedding model?	Domain/language fit, dimension, latency, cost, max length, eval on your data.	Changing it later?	Common industry pattern
27	★★	Metadata filtering and permissions?	Pre-filter at retrieval; ACL enforced in code.	Pre vs post filtering?	Common industry pattern
28	★★	Vector DB vs pgvector vs Elasticsearch?	Scale, ops, hybrid support, filtering, cost.	When is pgvector enough?	Common industry pattern
29	★★	Parent-child retrieval?	Match small, return big.	Token cost?	Common industry pattern
30	★	Embedding drift and re-indexing?	New model ⇒ re-embed all; versioned blue/green indexes.	Zero-downtime plan?	Common industry pattern
31	★	Multilingual retrieval?	Multilingual embeddings, language metadata, translate-query option.	Evaluate how?	Generated practice
32	★	MMR / diversity?	Trade relevance vs redundancy.	When harmful?	Generated practice

D. RAG architecture and variants (The core RAG pipeline, Better retrieval, All RAG types)

#	Priority	Question	Answer key	Follow-up	Provenance
33	★★★	Explain RAG end to end.	Ingest→chunk→embed→index; query→(rewrite)→retrieve→rerank→assemble→generate with citations.	Where does it fail most? See I2 in Intermediate.	Common industry pattern
34	★★★	RAG vs fine-tuning vs long context?	RAG for knowledge, fine-tune for behavior, long context for small corpora; they combine.	Cost/latency of each? See F10 in Fundamentals.	Common industry pattern
35	★★★	Bad answer despite correct doc in corpus — debug.	Indexed? retrieved? in final prompt? followed? Paste-gold-passage test.	Which logs? See S1 in Scenario Based.	Generated practice
36	★★★	Why does RAG still hallucinate?	Retrieval miss, noisy context, ignored context, ambiguous sources, no abstention.	Fixes? See A1 in Advanced.	Common industry pattern
37	★★	Naive vs advanced vs modular vs agentic RAG?	Maturity ladder; add complexity only for named failures.	Cost of agentic?	Common industry pattern
38	★★	HyDE?	Embed LLM-written hypothetical answer; bridges phrasing gap; may bias retrieval.	When avoid?	Common industry pattern
39	★★	GraphRAG vs vector RAG?	Graph for multi-hop/global questions; heavy indexing; staleness.	Example query?	Common industry pattern
40	★★	Self-RAG vs CRAG vs Adaptive RAG?	Self-critique tokens; grade-and-fallback; route by complexity.	Which is a drop-in?	Generated practice
41	★★	Multi-turn RAG?	Rewrite follow-ups to standalone query; history budget; skip retrieval for chit-chat.	Failure example?	Common industry pattern
42	★★	Query transformations?	Rewrite, multi-query, decomposition, step-back, HyDE; each +latency.	Gate them how?	Common industry pattern
43	★★	Citations and groundedness?	Chunk ids in prompt; require quotes; verify claims against sources.	Citation accuracy metric?	Common industry pattern
44	★★	Multimodal / table-heavy docs?	Layout-aware parsing, table preservation, VLM for charts, page citations.	See Multimodal GenAI and document AI.	Common industry pattern
45	★	Lost in the middle in RAG?	Put best chunks first/last, fewer better chunks.	Reorder strategy?	Common industry pattern
46	★	Contextual retrieval / chunk context enrichment?	Prepend doc/section context before embedding.	Cost at ingestion?	Generated practice
47	★	Freshness in RAG?	Incremental upserts, CDC, tombstones for deletes, freshness SLO.	Revoked docs?	Common industry pattern

E. Evaluation (RAG evaluation and hallucination, Observability and eval pipelines, Evaluating LLM systems, RAG evaluation in depth)

#	Priority	Question	Answer key	Follow-up	Provenance
48	★★★	How do you evaluate a RAG system?	Retrieval (recall@k, MRR, nDCG) separate from generation (faithfulness, relevance, correctness) + online signals.	Which first? See A7 in Advanced.	Common industry pattern
49	★★★	recall@k vs precision@k vs MRR vs nDCG?	Found-in-top-k; fraction relevant; 1/rank of first hit; graded rank-weighted.	Compute MRR for first-hit ranks 3, 1 and none.	Common industry pattern
50	★★★	Can you trust LLM-as-judge?	Biased (position, verbosity, self-preference); validate vs humans, rubric, pairwise both orders.	Agreement metric? See A12 in Advanced.	Common industry pattern
51	★★★	Build a golden set.	Real queries + SME cases, gold chunks, edge/unanswerable, versioned dev vs held-out.	Size/noise?	Common industry pattern
52	★★	How detect regressions after a prompt/model change?	Regression suite, paired comparison/CI, slices, gates, canary.	Small-sample noise?	Common industry pattern
53	★★	Offline vs online eval?	Offline gates; online A/B and implicit signals catch distribution shift.	Metrics?	Common industry pattern
54	★★	Benchmark contamination?	Test data in training inflates scores; use private product evals.	Detect how?	Common industry pattern
55	★★	Test abstention.	Include unanswerable queries; measure correct refusal vs false refusal.	Threshold choice? See A3 in Advanced.	Common industry pattern
56	★★	Evaluate a summarizer/chatbot.	Rubric (faithfulness, coverage, concision), deterministic checks, judge + human sample.	Slices?	Common industry pattern
57	★	Synthetic eval data pitfalls?	Easier, leaks wording, narrow; human-review a sample.	Mitigation?	Generated practice

F. Agents and tools (Function and tool calling, Agentic patterns: ReAct, planning and memory, Designing an agent product, Model Context Protocol and tool ecosystems, Multi-agent systems: when to use them and what they cost, Agent evaluation: state, trajectory, pass^k and release gates)

#	Priority	Question	Answer key	Follow-up	Provenance
58	★★★	How does function calling work?	Model emits structured call; app executes; result returned; model continues.	Who validates arguments? See I6 in Intermediate.	Common industry pattern
59	★★★	ReAct vs plan-and-execute?	Adaptive step-by-step vs upfront plan; hybrid with replanning.	Pick by? See A6 in Advanced.	Common industry pattern
60	★★★	Agent stuck in a loop.	Step/budget caps, repeated-call detection, progress checks, clear tool errors, escalate.	Root causes? See A2 in Advanced.	Generated practice
61	★★★	Single vs multi-agent?	Simplest wins; multi-agent for parallel breadth/isolation; costs tokens and coordination.	Example justification?	Common industry pattern
62	★★	What is MCP and how differs from function calling?	Standard client-server protocol (the host app embeds MCP clients) for tools/resources/prompts; function calling is the model-facing interface.	Security risks?	Common industry pattern
63	★★	Tool design best practices.	Clear names/descriptions, narrow scope, typed schemas, helpful errors, idempotency.	Too many tools?	Common industry pattern
64	★★	Human-in-the-loop.	Approval for irreversible/high-stakes actions; policy by risk tier.	Latency vs safety?	Common industry pattern
65	★★	Agent memory design.	Working/episodic/semantic/procedural; write/recall/forget; privacy.	Memory poisoning?	Common industry pattern
66	★★	Evaluate an agent.	State-based outcome, trajectory, pass^k reliability, cost/steps, safety.	Why pass^k?	Common industry pattern
67	★★	Error compounding.	pⁿ success; shorten chains, verify steps, checkpoints.	Compute 0.95¹⁰.	Common industry pattern
68	★	Context engineering.	Write/select/compress/isolate; cache-friendly layout.	Degrades after 30 steps?	Generated practice
69	★	Sandboxing code-executing agents.	Isolated containers, no secrets, egress allow-list, resource limits.	Escape risks?	Common industry pattern

G. Safety, security, guardrails (Guardrails, safety and prompt injection, Limiting blast radius and treating output as untrusted, Designing content moderation and guardrail systems)

#	Priority	Question	Answer key	Follow-up	Provenance
70	★★★	Prompt injection (direct vs indirect).	Instructions and data share a channel; indirect via docs/tools/web; defense in depth.	Why prompt alone fails? See I8 in Intermediate.	Common industry pattern
71	★★★	Defenses.	Least-privilege tools, authz in code, content isolation, output validation, egress control, approvals, monitoring.	Which is strongest? See A4 in Advanced.	Common industry pattern
72	★★	PII and data privacy.	Redact at ingestion/output, tenant isolation, retention, deletion across vectors/caches/logs.	Right to erasure?	Common industry pattern
73	★★	Guardrail architecture.	Input + output classifiers, policy engine, fallback/refusal, appeal path.	False positives?	Common industry pattern
74	★★	Jailbreak and red-teaming.	Adversarial test suites, automated attacks, regression set.	Frequency?	Common industry pattern
75	★★	OWASP LLM risks.	Prompt injection, sensitive information disclosure, improper output handling, excessive agency, supply chain, system prompt leakage (OWASP 2025 list; check the current list).	Pick two and mitigate.	Common industry pattern
76	★	Excessive agency.	Too many permissions/tools; scope and approvals.	Example incident?	Common industry pattern
77	★	Hallucination guardrails.	Grounding check, citation verification, abstain.	Cost? See A3 in Advanced.	Common industry pattern

H. Production, cost, latency, serving (Cost and latency optimization, LLM inference and serving, Production deployment of RAG, RAG security, reliability and release)

#	Priority	Question	Answer key	Follow-up	Provenance
78	★★★	LLM app too slow/expensive — what do you do?	Measure first (tokens, TTFT, stage latency), then caching, routing, shorter prompts, parallelism, smaller model for cost and latency; streaming only improves perceived latency; state the quality cost of each.	Order of attack? See I9 in Intermediate.	Common industry pattern
79	★★★	Semantic caching risks.	Wrong near-hit; scope by tenant/ACL/version; threshold; TTL/invalidation.	Measure false hits? See A5 in Advanced.	Generated practice
80	★★★	Model routing.	Classify difficulty; small model for easy; evaluate no quality loss.	Router errors?	Common industry pattern
81	★★	Prefill vs decode.	Compute-bound vs bandwidth-bound.	Which metric each?	Common industry pattern
82	★★	Continuous batching & PagedAttention.	Iteration-level scheduling; paged KV removes fragmentation.	Why raises throughput?	Common industry pattern
83	★★	Quantization trade-offs.	Memory/speed vs quality; validate on your eval.	Weight vs KV quantization?	Common industry pattern
84	★★	Speculative decoding.	Draft proposes, target verifies; same distribution; speedup by acceptance rate.	Downsides?	Common industry pattern
85	★★	Self-host vs API.	Cost at scale, data residency, control vs ops burden, quality.	Break-even?	Common industry pattern
86	★★	Streaming.	Perceived latency; TTFT.	Doesn’t reduce total time?	Common industry pattern
87	★★	Observability.	Trace prompts, retrieved ids, latency/cost per stage, feedback.	Privacy of logs?	Common industry pattern
88	★★	Rate limits and reliability.	Backoff, circuit breaker, fallback provider/model, queues.	Degradation order?	Common industry pattern
89	★	Release process.	Offline eval → shadow → canary → rollback; version prompts/models/indexes.	Provider model deprecations?	Common industry pattern

I. Fine-tuning (Fine-tuning vs RAG vs prompting, Fine-tuning methods: SFT, LoRA, QLoRA and DPO, Fine-tuning data, risks and shipping)

#	Priority	Question	Answer key	Follow-up	Provenance
90	★★★	When to fine-tune vs RAG vs prompt?	Prompt first, RAG for knowledge, fine-tune for behavior/format/distillation.	Does fine-tuning fix hallucination? See I10 in Intermediate.	Common industry pattern
91	★★★	Explain LoRA.	Frozen W + low-rank BA; tiny trainable fraction; mergeable/swappable.	Param fraction for r=8, d=4096?	Common industry pattern
92	★★	QLoRA.	4-bit base + LoRA adapters; fits large models on small GPUs.	Trade-off?	Common industry pattern
93	★★	RLHF vs DPO.	Reward model + RL vs direct preference loss.	Why DPO simpler?	Common industry pattern
94	★★	Data for fine-tuning.	Quality, diversity, dedupe, format match, holdout, decontamination.	How many examples?	Common industry pattern
95	★	Catastrophic forgetting.	Regression on general skills; mix data, lower LR, adapters, regression suite.	Detect?	Common industry pattern

J. Projects, judgment, and scenarios (deep dives in Scenario Based and Projects and Behavioral)

#	Priority	Question	Answer key	Follow-up	Provenance
96	★★★	Walk me through your best GenAI project.	Problem, constraints, architecture, your decisions, metrics (real), failures, what you’d change.	Why that design over alternatives?	Common industry pattern
97	★★★	Design an enterprise RAG assistant with permissions.	Requirements→ingestion→index→hybrid+ACL retrieval→generation→eval→security→cost→failure modes.	See Designing a RAG QA system.	Common industry pattern
98	★★	When would you NOT use an LLM?	Deterministic rules, SQL/APIs, tiny static data, strict latency/cost, high-stakes without verification.	Example? See A11 in Advanced.	Generated practice
99	★★	Quality dropped in production last week — triage.	Diff versions, check ingestion/index, traces by slice, replay eval, rollback.	What first?	Generated practice
100	★★	Trade-off: quality vs latency vs cost — how do you decide?	Tie to user value and SLOs; measure Pareto frontier; pick per-query routes.	Give a real example.	Generated practice

Study order (suggested)

Start with the ★★★ items: answer aloud, then compare.
Then ★★ items in the sections matching your next mock.
Take ★ items once the first two groups feel solid. Pair with: Fundamentals, Intermediate, Advanced, Scenario Based, Product Company, Coding Round, Projects and Behavioral, RAG interview drills and mock, and Coding, project defense and mock rounds.