Question bank
Interview Questions — Fundamentals
10 itemsWhat the provenance labels mean
In this bank
Round: GenAI/LLM depth (required) · Difficulty: below-level to at-level for a mid-level AI/GenAI engineer Provenance: each question is labelled in the table. None is verified company-specific. How to use: answer out loud in 60–90 s before opening the rubric. Then compare your answer with the points under Strong answer and note what you missed.
| # | Question | Difficulty | Provenance | Lesson |
|---|---|---|---|---|
| F1 | What does an LLM actually compute at each generation step? | below | Common industry pattern | How LLMs work |
| F2 | Explain self-attention including Q/K/V. | at | Common industry pattern | How LLMs work |
| F3 | Why did transformers replace RNNs? | below | Common industry pattern | How LLMs work |
| F4 | What is a KV-cache and why does it matter? | at | Common industry pattern | How LLMs work |
| F5 | What is tokenization; why isn’t a token a word? | below | Common industry pattern | Tokenization and embeddings |
| F6 | Explain temperature, top-k, top-p. | at | Common industry pattern | Sampling parameters and the model landscape |
| F7 | Zero-shot vs few-shot vs chain-of-thought prompting? | below | Common industry pattern | Prompting fundamentals |
| F8 | What is an embedding; why cosine similarity? | at | Common industry pattern | Embeddings and similarity search |
| F9 | What is RAG in two sentences? | below | Common industry pattern | RAG architecture and chunking |
| F10 | RAG vs fine-tuning? | at | Common industry pattern | Fine-tuning vs RAG vs prompting |
F1. What does an LLM compute at each generation step?
Strong answer includes: a probability distribution (logits → softmax) over the vocabulary for the next token, conditioned on all previous tokens; a sampling/decoding rule picks one; the token is appended and the loop repeats (autoregressive). Training objective = next-token cross-entropy. Follow-up: “Where does randomness come from, and how do you make output more repeatable?” (sampling; low temperature or greedy decoding makes it more repeatable, but hosted inference can still vary run to run because of batching and floating-point effects. See Sampling parameters and the model landscape.) Common mistake: “it predicts the next word” and stops — no distribution, no decoding step.
F2. Explain self-attention (Q/K/V)
Strong answer: each token is projected to a query, key and value. Score = Q·Kᵀ/√d_k against every position; softmax gives weights; output = weighted sum of V. Runs in parallel across positions; causal mask hides future tokens in decoder LLMs. Multi-head = several subspaces attended in parallel. Cost O(n²) in sequence length. Follow-up: “Why divide by √d_k?” (keeps dot-product variance ~1 so softmax doesn’t saturate and gradients don’t vanish.) “Why can’t attention alone tell dog bites man from man bites dog?” (without position information it treats the tokens as an unordered set → positional encodings.) Common mistake: describing attention as “the model looks at important words” without Q/K/V mechanics.
F3. Why transformers over RNNs?
Strong answer: parallel training over the sequence (RNNs are sequential in time), direct O(1)-path access between any two tokens (long-range dependencies vs vanishing gradients), scales well with data/compute. Cost: O(n²) attention memory/compute and no built-in recurrence/state. Follow-up: “Where do RNN/SSM-style models still make sense?” (very long streaming sequences, constant-memory inference.) Common mistake: saying ‘transformers are just faster’ and ignoring the O(n^2) attention cost.
F4. KV-cache
Strong answer: during decoding, K and V for earlier tokens never change, so cache them and compute K/V only for the new token → per-step cost drops from recomputing everything to attending over cached state. Cost: memory grows linearly with sequence length × layers × KV heads × batch, so it caps concurrency and long-context serving. Prefill (prompt) is compute-bound; decode is memory-bandwidth-bound. Follow-up: “Why doesn’t the KV-cache help the first token?” (the prompt still has to be processed once — prefill.) “How do GQA/MQA help?” (fewer K/V heads → smaller cache.) Common mistake: saying the KV-cache stores outputs or responses (that is a response cache — different thing).
F5. Tokenization
Strong answer: text → subword units (BPE/WordPiece/SentencePiece) → integer IDs; balances vocabulary size vs sequence length, handles rare/unseen words without UNK. Consequences: cost/latency/context billed in tokens; non-English text and code can cost more tokens; arithmetic, spelling and character-level tasks are error-prone because the model never sees characters directly. Follow-up: “Why can’t you compare token counts across model families?” (different tokenizers.) Common mistake: treating a token as a word, or as a fixed number of characters for every language.
F6. Temperature, top-k, top-p
Strong answer: temperature rescales logits before softmax (low = sharper and more repeatable, high = flatter and more diverse; T→0 approaches greedy decoding). Top-k keeps the k most likely tokens. Top-p (nucleus) keeps the smallest set whose cumulative probability ≥ p — adapts to confidence. Typical: low T for extraction/classification/code, higher for creative; tune temperature or top-p, rarely both. Follow-up: “Does temperature 0 guarantee identical outputs?” (No. Greedy decoding is repeatable in theory, but real serving stacks can still differ between runs. See F1.) Common mistake: tuning temperature and top-p together without measuring, or assuming temperature 0 gives identical output every time.
F7. Zero-shot / few-shot / CoT
Strong answer: zero-shot = instruction only; few-shot = in-context examples that fix format and behaviour (cost: tokens, example bias); CoT = elicit intermediate reasoning, helps multi-step problems, costs latency/tokens, and rationales can be unfaithful. Reasoning-tuned models may not need an explicit “think step by step.” Follow-up: “How do you pick few-shot examples?” (diverse, representative, include edge cases and negatives; test order sensitivity.) Common mistake: assuming ‘think step by step’ always helps, or picking few-shot examples by feel with no eval set.
F8. Embeddings and cosine similarity
Strong answer: dense vectors where semantic closeness ≈ geometric closeness, trained contrastively. Cosine compares direction not magnitude; on unit-normalised vectors it equals dot product (cheap, and a common choice in vector indexes). Limits: embeddings blur exact identifiers/numbers, domain shift hurts, asymmetric query-vs-document phrasing. Follow-up: “When would you prefer dot product or L2?” (when the model was trained for it / magnitude carries signal.) Common mistake: believing embeddings capture exact-match meaning, or comparing vectors from two different embedding models.
F9. RAG in two sentences
Strong answer: retrieve relevant external chunks at query time and put them in the prompt so the model answers grounded in that evidence; no weight updates, knowledge can be refreshed by re-indexing and answers can cite sources. It reduces but does not eliminate hallucination. Common mistake: claiming RAG eliminates hallucination, or describing it without the retrieval step at query time.
F10. RAG vs fine-tuning
Strong answer: RAG = knowledge (fresh, private, citable, permissionable). Fine-tuning = behaviour (style, format, task skill, latency via smaller model). Fine-tuning is a poor, expensive way to inject changing facts. Often combine: RAG for facts + light fine-tune for format/tone. Decision order: prompt → RAG → fine-tune. Follow-up: “Your data changes daily and users need citations — which, and why?” (RAG.) Common mistake: proposing fine-tuning to inject facts that change often, or treating RAG and fine-tuning as either/or.