Reference

Glossary

239 terms used in the lessons, each linked back to where it is explained.

A

A/B test: An online experiment that sends different users to two versions and compares real-use signals such as edits, retries and abandonment. It measures real use, but it is noisy and arrives after release.; Explained in Evaluating LLM systems
Abstention: A system declining to answer, or asking a clarifying question, when retrieval is empty or weak. Calibrate the gate on labelled data.; Explained in RAG evaluation and hallucination RAG security, reliability and release
Access control list (ACL): The record of who may read a document. In RAG it is stored as metadata and enforced inside retrieval, not left to the prompt.; Explained in RAG architecture and chunking Production deployment of RAG RAG security, reliability and release
ACLAbbreviation: Short for access control list: the record of who may read a document, enforced inside retrieval.; See Access control list (ACL)Explained in RAG architecture and chunking Production deployment of RAG RAG security, reliability and release
Adaptive RAG: A RAG variant that routes each query to no retrieval, single-shot retrieval or multi-step retrieval.; Explained in All RAG types
Agent: A system in which the model decides its own steps and tools in a loop until a goal is met. It costs more than a fixed workflow and errors can compound.; Explained in Agentic patterns: ReAct, planning and memory
Agent memory: State kept by an agent. The course splits it into four types: working, episodic, semantic and procedural. Short-term memory is the current context and scratchpad; long-term memory is stored outside and retrieved when relevant.; Explained in Agentic patterns: ReAct, planning and memory Context engineering and memory
Agentic RAG: A RAG design in which the model decides whether, what and how often to retrieve. It needs step limits, tracing and guardrails, and its cost and latency vary.; Explained in All RAG types
ANNAbbreviation: Short for approximate nearest neighbor search: finding very close, not always the closest, vectors with an index, trading some recall for speed and memory.; See Approximate nearest neighbor (ANN) searchExplained in Vector databases
Answer relevance: Whether the answer addresses the question that was asked.; Explained in RAG evaluation in depth
API-only model: A model you reach only through a provider's API. No infrastructure to run and often top quality, but your data goes to the provider and you cannot change the weights.; Explained in Sampling parameters and the model landscape
Approximate nearest neighbor (ANN) search: Finding very close, though not always the closest, vectors with an index. You trade some recall for lower latency and memory.; Explained in Vector databases
Autoregressive generation: Producing text one token at a time, where each new token is appended to the input before the next step. This is why output length drives latency.; Explained in How LLMs work

B

Base rate: How common the thing you are detecting is. When the false-positive rate is not far below the base rate, most blocks are false positives.; Explained in Designing content moderation and guardrail systems
Batch API: A provider interface that runs requests asynchronously at lower cost in exchange for delayed results.; Explained in Cost and latency optimization
Batch invariance: The property that a request's result does not depend on which other requests share its batch. Many inference kernels lack it, which is one reason temperature 0 is not fully deterministic.; Explained in Sampling parameters and the model landscape
Benchmark contamination: Benchmark content having leaked into a model's training data, which inflates its scores. Use public benchmarks to shortlist, not to decide.; Explained in Evaluating LLM systems
Best-of-N: Generating N answers and keeping the one a checker or reward model scores highest. It spends more inference compute for a better pick.; Explained in Reasoning models and structured outputs
Bi-encoder: A model that embeds query and document separately, so document vectors can be precomputed. Fast, used for first-stage retrieval.; Explained in Better retrieval
Blast radius: How much damage one failure or compromise can cause. Autonomy and permissions should match it.; Explained in Limiting blast radius and treating output as untrusted Designing an agent product
BM25: A keyword ranking function that scores documents by term frequency, term rarity and length. It catches exact terms and identifiers that embeddings miss.; Explained in Embeddings and similarity search Better retrieval
Bootstrap: Resampling the test cases with replacement many times and recomputing a metric each time, to get an uncertainty interval without a formula.; Explained in Evaluating LLM systems
BPEAbbreviation: Short for byte-pair encoding: building a subword vocabulary by repeatedly merging the most frequent adjacent pair.; See Byte-pair encoding (BPE)Explained in Tokenization and embeddings
Byte-pair encoding (BPE): A way to build a subword vocabulary by repeatedly merging the most frequent adjacent pair of symbols. Any text, even an unseen word, can then be written from a fixed set of pieces.; Explained in Tokenization and embeddings

C

Canary release: Sending a small share of real traffic to a new version before full rollout, with a way to roll back.; Explained in RAG security, reliability and release
Causal mask: A rule in attention that blocks each token from looking at later tokens. Without it the model could copy the answer it is supposed to predict.; Explained in How LLMs work
Chain-of-thought (CoT): Having the model write intermediate reasoning steps before the answer. It can help multi-step problems, but it costs tokens and latency, and the written reasoning may not be what drove the answer.; Explained in Prompting fundamentals
Chat template: The exact message format a model was trained with. Training on a different format from production is a common cause of a model that looks good offline and worse live.; Explained in Fine-tuning data, risks and shipping
Chunk: A piece of a document that is embedded and retrieved as one unit. Its size and boundaries strongly affect retrieval quality.; Explained in RAG architecture and chunking
Chunk overlap: Repeating a bit of text at chunk boundaries so facts there stay whole in at least one chunk. The cost is more, near-duplicate chunks.; Explained in RAG architecture and chunking
Chunked prefill: Splitting a long prompt's prefill into pieces so other users' decode steps keep running. It smooths streaming for others at some cost to the long prompt's first token.; Explained in LLM inference and serving
Chunking: Splitting documents into chunks before indexing. Small chunks match precisely but lose context; large chunks keep context but blur the match.; Explained in RAG architecture and chunking
Citation: A pointer from a claim in the answer to its source chunk. Its presence does not prove the source supports the claim, so check citation accuracy.; Explained in RAG evaluation and hallucination
Cohen's kappa: A measure of agreement between two raters, such as judge and human, corrected for agreement expected by chance. Raw percent agreement misleads when one label dominates.; Explained in Evaluating LLM systems
Compensating action: A step that undoes an earlier side effect, such as cancelling a hold, when a database transaction cannot roll it back. Run them in reverse order on failure; some actions have no undo and need approval.; Explained in Designing an agent product
Constrained decoding: Restricting which tokens the model may produce at each step so the output must fit a schema. The shape conforms; the content can still be wrong.; Explained in Reasoning models and structured outputs
Context assembly: The step before generation that de-duplicates, orders and trims retrieved chunks to a token budget and attaches source ids.; Explained in Better retrieval
Context compaction: Summarising or trimming earlier context to free space. Must-keep rules should be pinned verbatim so a summary cannot drop them.; Explained in Context engineering and memory
Context distraction: In this course's taxonomy, lower accuracy caused by extra, irrelevant context. Measure accuracy with and without it.; Explained in Context engineering and memory
Context drift: In this course's taxonomy, loss of a constraint or fact after repeated summarising. Test with probe questions after each compaction.; Explained in Context engineering and memory
Context engineering: Deciding what goes into the context window at each step. One common grouping (LangChain) names four moves: write, select, compress and isolate.; Explained in Context engineering and memory
Context overflow: In this course's taxonomy, the context growing past the window so calls fail or the oldest content is silently cut. Budget tokens per section and compress or isolate earlier work.; Explained in Context engineering and memory
Context precision: A RAGAS-style retrieval metric for whether the relevant retrieved context ranks high in the list.; Explained in RAG evaluation in depth
Context recall: A RAGAS-style retrieval metric for whether the retrieved context covers what the gold answer needs.; Explained in RAG evaluation in depth
Context window: The maximum number of tokens a model can take in and produce in one request. A larger window is not free: you pay per token, and attention work and KV-cache memory grow with length.; Explained in Sampling parameters and the model landscape
Contextual retrieval: A chunking technique where a model writes a short note placing each chunk in its document, and the note is added to the chunk before embedding and BM25 indexing. It costs one extra model call per chunk at ingest.; Explained in RAG architecture and chunking Better retrieval
Continuous batching: Scheduling work per decode step so finished requests leave a batch and new ones join immediately.; Explained in LLM inference and serving
Contrastive training: Training that pulls matching pairs of texts together and pushes others apart in vector space. It is why close vectors tend to mean similar meaning.; Explained in Embeddings and similarity search
Corrective RAG (CRAG): A RAG variant that grades retrieved documents and falls back, for example by rewriting the query or searching the web, when they are weak.; Explained in All RAG types
Cosine similarity: A measure of the angle between two vectors, ignoring their length. It is the usual default for text embeddings, provided the index uses the metric the model expects.; Explained in Embeddings and similarity search
CoTAbbreviation: Short for chain-of-thought: having the model write intermediate reasoning steps before the answer.; See Chain-of-thought (CoT)Explained in Prompting fundamentals
CRAGAbbreviation: Short for corrective RAG: a variant that grades retrieved documents and falls back when they are weak.; See Corrective RAG (CRAG)Explained in All RAG types
Cross-encoder: A model that reads query and passage together and outputs a relevance score. More accurate, but one call per pair, so it reranks a shortlist.; Explained in Better retrieval

D

Decode: The phase after prefill where one token is generated per step. Each step has to read the weights and the KV cache, so it is usually limited by memory bandwidth.; Explained in How LLMs work LLM inference and serving
Defence in depth: Stacking several imperfect controls so one failure is not fatal: prompts, data marking, screening, moderation, output checks, least privilege and monitoring.; Explained in Guardrails, safety and prompt injection
Delimiter: A marker such as a tag or fence that separates instructions from untrusted data in a prompt. It lowers injection risk but does not remove it.; Explained in Prompting fundamentals
Deterministic check: A rule-based test such as schema validity, a regex or unit tests. It is fast, free and consistent.; Explained in Evaluating LLM systems
Deterministic chunk ID: A chunk id that comes out the same each time the same input is ingested, such as the document id plus the chunk number or a hash of the chunk's text. Re-running ingestion then updates chunks instead of duplicating them. Old chunks of an edited or shortened document still have to be deleted, because a changed chunk can get a new id.; Explained in Vector databases Production deployment of RAG
Dev set: The data you iterate on while building.; Explained in Evaluating LLM systems
Direct prompt injection: An attack typed by the user into their own input.; Explained in Guardrails, safety and prompt injection
Distillation: Training a small model on a large model's outputs to keep a narrow task's quality at lower cost and latency.; Explained in Fine-tuning methods: SFT, LoRA, QLoRA and DPO
Dot product: A similarity score that multiplies matching components and adds them. It mixes angle with length, so use the metric the embedding model was trained for; on unit-length vectors it equals cosine.; Explained in Embeddings and similarity search
DPO: Direct preference optimization: trains directly on preference pairs, without a separate reward model or reinforcement-learning loop. It still uses a frozen reference copy of the model.; Explained in Fine-tuning methods: SFT, LoRA, QLoRA and DPO

E

ef_search: The HNSW query setting for how many candidates are tracked per query. Higher values raise recall and latency, and it can be changed per query.; Explained in Vector databases
Embedding: A list of numbers (a vector) that represents a token, chunk or query so that similar meanings sit close together. Used for search, clustering and as the model's input.; Explained in Tokenization and embeddings Embeddings and similarity search
Embedding model: A model whose job is to turn text into one fixed-length vector. Index and query must use the same one, because each model builds its own vector space.; Explained in Embeddings and similarity search
Embedding table: The learned matrix with one row per token in the vocabulary. Looking up a token id returns its row, and the table is trained together with the rest of the model.; Explained in Tokenization and embeddings
Episodic memory: Timestamped records of past interactions, often stored as summaries in a vector store.; Explained in Context engineering and memory
Error analysis: Reading real failing traces, sorting them into categories and measuring the top ones before choosing metrics.; Explained in Evaluating LLM systems
Error compounding: If each step succeeds with probability p, independently, n steps all succeed with probability p to the power n, so long chains fail more than single steps suggest.; Explained in Multi-agent systems: when to use them and what they cost
Euclidean distance: The straight-line distance between two vectors. On unit-length vectors it ranks results the same as cosine.; Explained in Embeddings and similarity search
Exact match and F1: Reference-based metrics for tasks with a gold answer, such as extraction. Exact match requires an identical answer; F1 scores partial overlap.; Explained in Evaluating LLM systems
Exact search: Comparing the query with every stored vector to get the true nearest neighbours. Cost grows with vectors times dimensions, so it stops working at large scale.; Explained in Vector databases
Excessive agency: Harm from an agent that has too many tools, too much permission or too much autonomy.; Explained in Limiting blast radius and treating output as untrusted
Executor: The application code that receives a model-proposed tool call and runs it. It must validate arguments, authorize from the real session, apply timeouts and treat results as untrusted.; Explained in Function and tool calling

F

Faithfulness: Whether an answer's claims are supported by the retrieved context. It is a generation metric, separate from whether the answer is correct.; Explained in RAG evaluation in depth
False negative: In a guardrail or classifier, an attack or harmful item that passes.; Explained in Guardrails, safety and prompt injection
False positive: In a guardrail or classifier, a legitimate request or item that is blocked or flagged.; Explained in Guardrails, safety and prompt injection
Few-shot prompting: Putting a few worked examples in the prompt so the model copies the pattern. No weights change, but you pay for the example tokens on every call.; Explained in Prompting fundamentals
Field-level accuracy: The share of individual extracted fields, such as a total or a date, that match labelled values. It is a core metric for document extraction.; Explained in Multimodal GenAI and document AI
Fine-tuning: Updating a model's weights on labelled examples to teach a consistent behaviour, style or format. It is a poor way to add or update facts.; Explained in Fine-tuning vs RAG vs prompting Fine-tuning methods: SFT, LoRA, QLoRA and DPO Fine-tuning data, risks and shipping
FlashAttention: A way of computing attention that reduces reads and writes to slow GPU memory, giving the same result faster.; Explained in LLM inference and serving
Forgetting (catastrophic forgetting): The loss of general abilities a model was not trained to keep after fine-tuning. Mix in general data and re-run regression tests.; Explained in Fine-tuning data, risks and shipping
Function calling (tool calling): The model emits a structured request with a tool name and arguments, and your code runs it and returns the result. The model never executes anything itself.; Explained in Function and tool calling

G

Golden set: A curated set of questions with known good answers or sources, used to score a system on every change. Keep slices tagged and add real failures over time.; Explained in RAG evaluation in depth Observability and eval pipelines
Goodput: The rate of requests that meet the latency targets. Throughput can look high while many users get an unacceptable experience. DistServe uses a stricter version: the highest request rate at which a target share of requests meets the limits.; Explained in LLM inference and serving
GQAAbbreviation: Short for grouped-query attention: query heads share key and value heads, which shrinks the KV cache.; See Grouped-query attention (GQA)Explained in How LLMs work LLM inference and serving
Graceful degradation: Falling back to a simpler, known-quality path when a component fails, such as skipping the reranker. Never drop permission filtering.; Explained in RAG security, reliability and release
GraphRAG: A RAG variant that builds a graph of entities and relations from the corpus. It suits corpus-wide (global) questions and multi-hop ones, with costly indexing and a graph that can go stale.; Explained in All RAG types
Greedy decoding: Always picking the single most likely token. It is what temperature 0 amounts to, and it still does not guarantee identical outputs run to run.; Explained in Sampling parameters and the model landscape
Grounding: Tying an answer to supplied sources, such as retrieved passages, so claims can be checked.; Explained in RAG evaluation and hallucination
Grouped-query attention (GQA): An attention variant in which several query heads share one set of key and value heads. Fewer KV heads means a smaller KV cache. Multi-query attention (MQA) is the extreme case with one KV head.; Explained in How LLMs work LLM inference and serving
Guardrail: A check around a model, on input, output or actions. It reduces risk but does not solve prompt injection.; Explained in Guardrails, safety and prompt injection

H

Hallucination: Fluent output that is not supported by the source or by fact. RAG reduces it by grounding answers, but does not eliminate it.; Explained in RAG evaluation and hallucination
Held-out set: Data kept aside to gate releases. If you tune against it, it becomes another dev set and its score stops meaning anything.; Explained in Evaluating LLM systems
HNSW: Hierarchical Navigable Small World, a layered graph index. Search starts in a sparse top layer and descends to a dense bottom layer. High recall, memory-heavy.; Explained in Vector databases
Hosted tool: A tool such as web search or code execution that runs on the provider's infrastructure. Less code for you, less control, and possible extra charges.; Explained in Function and tool calling
Human approval: Requiring a person to confirm an irreversible, high-value or external action, such as payments or deletions, before it runs.; Explained in Limiting blast radius and treating output as untrusted Designing an agent product
Hybrid search: Running keyword search (BM25) and vector search together and merging the lists. It covers exact terms and paraphrases.; Explained in Better retrieval
HyDE: Hypothetical document embeddings: the model writes a plausible answer and that text is used to search. It helps when queries and documents are phrased differently, but a wrong draft can bias retrieval.; Explained in All RAG types

I

Idempotency key: A unique value sent with a side-effecting request so the receiver can recognise a repeat and not act twice. It protects against retries and loops.; Explained in Function and tool calling Designing an agent product
Improper output handling: Passing model output to a browser, database or shell without validating or encoding it. Treat output as untrusted input.; Explained in Limiting blast radius and treating output as untrusted
Index alias: A name that points to the live index version. Swapping it switches versions at once and allows one-step rollback.; Explained in Production deployment of RAG
Indirect prompt injection: An attack hidden in third-party content the model reads, such as a web page, email, retrieved document or tool result.; Explained in Guardrails, safety and prompt injection
Ingestion: The offline part of a RAG system: parse, chunk, embed and index documents. Safe ingestion is idempotent, incremental and records model and chunker versions.; Explained in The core RAG pipeline Production deployment of RAG
Inter-token latency: The gap between successive output tokens (also called time per output token, TPOT), which sets how smooth streaming feels. It reflects decode speed and batch pressure.; Explained in LLM inference and serving
IVF: Inverted file index: vectors are grouped into clusters around learned centres, and a query scans only the nearest clusters. Lighter on memory than HNSW, and needs training on representative data.; Explained in Vector databases

J

Jailbreak: A direct attack aimed at getting the model to break its own safety rules.; Explained in Guardrails, safety and prompt injection
JSON mode: A setting that makes the model return syntactically valid JSON. It does not guarantee the keys or types match your schema.; Explained in Reasoning models and structured outputs
JSON Schema: A standard way to describe the shape and types of JSON. Providers use it to define tool arguments and structured outputs, and support only a subset of it.; Explained in Reasoning models and structured outputs Function and tool calling

K

KV cache: Stored key and value vectors of earlier tokens, reused at each decode step so they are not recomputed. It saves compute but its memory grows with context length and concurrent requests. It is not a response cache.; Explained in How LLMs work LLM inference and serving

L

Large language model (LLM): A neural network trained on very large amounts of text to predict the next token. Chaining that prediction step by step lets it write, answer and follow instructions.; Explained in How LLMs work
Latency budget: An end-to-end time target split among stages such as retrieval, rerank and generation, so each stage has a limit.; Explained in Production deployment of RAG
Layout analysis: Detecting the structure of a page, such as columns, tables, headings and reading order, so extracted text keeps its meaning.; Explained in Multimodal GenAI and document AI
Least privilege: Giving each agent and tool only the access its task needs, with read kept apart from write or send, so an attack has a small blast radius.; Explained in Limiting blast radius and treating output as untrusted
Lethal trifecta: The combination of private data, exposure to untrusted content and an outbound channel in one agent. Together they allow data theft, so do not combine all three.; Explained in Limiting blast radius and treating output as untrusted
LLMAbbreviation: Short for large language model: a network trained on large amounts of text to predict the next token.; See Large language model (LLM)Explained in How LLMs work
LLM-as-judge: Using a model to score outputs against a rubric or pick the better of two. It scales, but has its own biases and must be validated against human labels.; Explained in RAG evaluation in depth Evaluating LLM systems
Logits: The raw scores the model gives each vocabulary token at a step, before they become probabilities. Sampling settings such as temperature act on them.; Explained in How LLMs work Sampling parameters and the model landscape
LoRA: Low-rank adaptation: freeze the base weights and train a small low-rank update, a tiny fraction of the parameters. Adapters are small, swappable and mergeable.; Explained in Fine-tuning methods: SFT, LoRA, QLoRA and DPO
Lost in the middle: The finding that models use information in the middle of a long context less reliably than information at the start or end.; Explained in Better retrieval

M

Maximal marginal relevance (MMR): A selection method that balances relevance against redundancy, so the top-k results are not near-duplicates.; Explained in Better retrieval
MCPAbbreviation: Short for Model Context Protocol: an open protocol connecting a host application to servers that expose tools, resources and prompts.; See Model Context Protocol (MCP)Explained in Model Context Protocol and tool ecosystems
MCP client: The component inside an MCP host that keeps one connection to one MCP server. The host runs one client per server and relays tool lists and calls between the model and that server.; Explained in Model Context Protocol and tool ecosystems
MCP host: The AI application, such as an IDE or chat app, that holds one MCP client per server and shows the servers' tools to the model through tool calling. The model never talks to a server directly; the host mediates, so approvals, logging and limits live there.; Explained in Model Context Protocol and tool ecosystems
MCP server: A program that exposes tools, resources and prompts through MCP. Connecting one means trusting text it controls.; Explained in Model Context Protocol and tool ecosystems
MCP tools, resources and prompts: The three things a server offers: tools are actions the model invokes, resources are context identified by URI, and prompts are templates the user picks.; Explained in Model Context Protocol and tool ecosystems
Mean reciprocal rank (MRR): The average of 1/rank of the first relevant result across queries, counting 0 for a miss. It rewards ranking the right item higher.; Explained in RAG evaluation in depth
Memory-bandwidth-bound: Limited by how fast data can be read from memory rather than by arithmetic. Decode is typically this.; Explained in LLM inference and serving
Metadata: Data about a chunk such as source, section, page, date or access rights. It supports filtering, citations and permission checks.; Explained in RAG architecture and chunking
Metadata filter: A condition on fields such as department, date or permission applied to a vector search. Applied after the search it can return fewer than k results. See post-filtering.; Explained in Vector databases Better retrieval
MMRAbbreviation: Short for maximal marginal relevance: choosing results that are relevant and different from those already chosen.; See Maximal marginal relevance (MMR)Explained in Better retrieval
Model Context Protocol (MCP): An open protocol through which a host application connects to servers that expose tools, resources and prompts. It turns N x M custom integrations into N + M.; Explained in Model Context Protocol and tool ecosystems
Model routing: Sending easy requests to a small, cheap model and hard ones to a stronger one. It saves cost when most requests stay on the small model; check quality per query type.; Explained in Cost and latency optimization
Moderation: Detecting harmful content such as hate or violence. It is not an injection detector.; Explained in Guardrails, safety and prompt injection Designing content moderation and guardrail systems
MRRAbbreviation: Short for mean reciprocal rank: the average of 1/rank of the first relevant result, counting 0 for a miss.; See Mean reciprocal rank (MRR)Explained in RAG evaluation in depth
Multi-agent system: Several coordinating agents. Justified for broad, parallel work or separate permissions; otherwise it adds tokens, latency and failure modes.; Explained in Multi-agent systems: when to use them and what they cost
Multi-head attention: Several attention computations run in parallel (heads), each free to learn a different relationship such as syntax or coreference. Their outputs are concatenated.; Explained in How LLMs work
Multi-tenancy: One system serving many customers whose data must stay separate. Permissions, caches and indexes must be scoped by tenant.; Explained in Production deployment of RAG RAG security, reliability and release

N

nDCG: Normalized discounted cumulative gain: a ranking metric that rewards putting highly relevant items earlier and handles graded relevance, not just relevant or not.; Explained in RAG evaluation in depth
nprobe: The IVF query setting for how many clusters to scan. More probes raise recall and latency.; Explained in Vector databases

O

Observability: Recording enough about each request, such as versions, inputs, retrieved ids, tokens, latency and cost, to see what happened and replay failures.; Explained in Observability and eval pipelines
OCRAbbreviation: Short for optical character recognition: software that reads text from images of pages.; See Optical character recognition (OCR)Explained in Multimodal GenAI and document AI
Offline evaluation: Running a fixed test set against a change before release. It catches regressions, including rare intents.; Explained in Observability and eval pipelines
Online monitoring: Watching live traffic for errors, latency, cost and drift. It catches what offline tests cannot see.; Explained in Observability and eval pipelines
Open-weight model: A model whose weights you can download. You can self-host and fine-tune it and keep data in your infrastructure, but you run serving and operations and must read the license.; Explained in Sampling parameters and the model landscape
Optical character recognition (OCR): Software that reads text from images of pages. It is one step in reading scans, often paired with layout analysis.; Explained in Multimodal GenAI and document AI
Outcome grading: Checking the state of the world after an agent runs, such as the database row or passing tests, instead of trusting its final message.; Explained in Agent evaluation: state, trajectory, pass^k and release gates
Overfitting: A model memorising its training examples and generalising worse.; Explained in Fine-tuning data, risks and shipping

P

Page-image retrieval: Embedding page images directly and retrieving whole pages, skipping text parsing. It sees layout, but the index is larger and the generator still has to read the page.; Explained in Multimodal GenAI and document AI
PagedAttention: Storing the KV cache in on-demand blocks to avoid fragmentation, which allows larger batches and prefix sharing.; Explained in LLM inference and serving
Parent-child retrieval: Matching on small chunks and then returning their larger parent section. It gives precise matching with more context, at the cost of bigger prompts.; Explained in RAG architecture and chunking Better retrieval
Parsing: Turning a source file such as a PDF into clean text and structure. It is the usual cap on document RAG, because a fact lost here cannot be recovered later.; Explained in Multimodal GenAI and document AI
pass@k: The chance that at least one of k runs succeeds. It hides run-to-run variance.; Explained in Agent evaluation: state, trajectory, pass^k and release gates
pass^k: The chance that all k runs succeed. It is the right measure for an agent that must work every time.; Explained in Agent evaluation: state, trajectory, pass^k and release gates
Percentile latency (p50, p95): The time under which 50 percent or 95 percent of requests finish. The tail (p95) shows what the slower users experience, which averages hide.; Explained in Cost and latency optimization
pgvector: An extension that adds vector search to PostgreSQL. A common first choice when you already run Postgres.; Explained in Vector databases
Pipeline parallelism: Splitting a model by layers, so different GPUs hold different whole layers and each passes its output to the next. It also lets a model too large for one GPU run, usually with less communication between GPUs than tensor parallelism needs.; Explained in LLM inference and serving
Plan-then-execute: An agent pattern that writes a plan first and then carries it out. It is predictable and reviewable, but brittle when the world deviates.; Explained in Agentic patterns: ReAct, planning and memory
Position bias: A judge's tendency to favour an answer because of where it appears in the prompt. Swapping the order in pairwise comparisons helps.; Explained in RAG evaluation in depth
Positional encoding: Extra information that tells the model where each token sits in the sequence. Attention alone has no sense of order, so position must be injected.; Explained in How LLMs work
Post-filtering: Applying a metadata filter after the nearest-neighbour search. Filtered-out candidates leave fewer than k results. The alternative is filtering inside the search; see metadata filter.; Explained in Vector databases RAG architecture and chunking
Precision@k: The fraction of the top k retrieved items that are relevant.; Explained in RAG evaluation in depth
Preference tuning: Fine-tuning on a prompt with a preferred and a rejected response, to teach what people prefer.; Explained in Fine-tuning methods: SFT, LoRA, QLoRA and DPO
Prefill: The first phase of a request, in which the whole prompt is processed in parallel. It sets time to first token and is usually limited by compute.; Explained in How LLMs work LLM inference and serving
Prefill/decode disaggregation: Running prefill and decode on separate GPU pools so the two phases do not interfere. It costs more machines and KV transfer.; Explained in LLM inference and serving
Prefix caching: Reusing the KV cache of a prompt prefix shared by requests, so the shared part is not recomputed. The serving-engine counterpart of a provider's prompt caching.; Explained in LLM inference and serving
Pretraining: The first, large training stage: next-token prediction over a huge text corpus. Most language ability and world knowledge come from it.; Explained in Fine-tuning methods: SFT, LoRA, QLoRA and DPO
Procedural memory: Stored instructions or skills that tell the agent how to do things, such as instruction files.; Explained in Context engineering and memory
Product quantization (PQ): A vector compression method that splits each vector into sub-vectors and stores, for each, the id of its nearest entry in a small learned codebook. It saves memory and speeds up distance checks, at some loss of recall.; Explained in Vector databases
Prompt: The full input sent to a model: instructions, context, examples and the user's request. Its wording and structure change the output, so changes should be measured.; Explained in Prompting fundamentals
Prompt caching: A provider feature that reuses the computed state of an identical prompt prefix. Cache reads cost a fraction of normal input and the first token comes sooner, while cache writes can cost a premium. Put static content first. Not the same as prefix caching, which happens inside your own serving engine.; Explained in Context engineering and memory Cost and latency optimization
Prompt injection: Text that changes what a model does in a way its operator did not intend. No guardrail fully prevents it; the aim is to limit what a hijacked model can do.; Explained in Guardrails, safety and prompt injection

Q

QLoRA: LoRA on top of a base model held in 4-bit precision, which cuts memory a lot at the cost of slower steps. Check quality on your task.; Explained in Fine-tuning methods: SFT, LoRA, QLoRA and DPO
Quantization: Storing numbers with fewer bits. For vectors it saves memory; for model weights it cuts memory and the bytes read per decode step. Both cost some rounding error, so test on your own eval set.; Explained in Vector databases LLM inference and serving
Query transformation: Rewriting a query before retrieval, for example with multi-query, decomposition or step-back. Each adds a model call and can drift from the user's intent.; Explained in Better retrieval
Query, key and value (Q, K, V): Three vectors computed for each token in attention. Queries are compared with keys to get weights, and the weights mix the values.; Explained in How LLMs work

R

RAGAbbreviation: Short for retrieval-augmented generation: putting retrieved content in the prompt so the answer is grounded in it.; See Retrieval-augmented generation (RAG)Explained in RAG architecture and chunking The core RAG pipeline
Re-embedding: Recomputing all vectors with a new embedding model. Build a new index version in the background, test it, then swap an alias.; Explained in Production deployment of RAG
ReAct: An agent loop of thought, action (a tool call) and observation (the result), repeated until done or a stopping control fires.; Explained in Agentic patterns: ReAct, planning and memory
Reasoning model: A model trained to spend extra output on working through a problem before answering. It helps hard multi-step tasks and costs more time and tokens.; Explained in Reasoning models and structured outputs
Reasoning tokens: Tokens a reasoning model generates while thinking. They are billed as output and count toward the output limit even when the API does not return them.; Explained in Reasoning models and structured outputs
Recall@k: The share of the relevant or true top-k items that appear in the top k results. For an ANN index it measures the recall lost to approximation.; Explained in Vector databases RAG evaluation in depth
Reciprocal rank fusion (RRF): A way to merge ranked lists by summing 1/(k + rank) per list, with k often 60. It uses ranks only, so incompatible score scales need no calibration.; Explained in Better retrieval Coding, project defense and mock rounds
Red-teaming: Attacking your own system with realistic attempts before release to find what to fix.; Explained in Guardrails, safety and prompt injection
Refusal: A response in which the model declines to answer. Code that parses structured output has to handle it, as it may not match the schema.; Explained in Reasoning models and structured outputs
Reranking: Re-ordering a shortlist of retrieved candidates with a more accurate model. It cannot recover a chunk the first stage failed to return.; Explained in Better retrieval
Retrieval-augmented generation (RAG): Retrieving relevant external content and putting it in the prompt so the answer is grounded in it, without changing model weights. Good for fresh, citable knowledge.; Explained in RAG architecture and chunking The core RAG pipeline
Reward model: A model that scores outputs by how well they match preferences. It is used in RLHF and to pick the best of several answers.; Explained in Fine-tuning methods: SFT, LoRA, QLoRA and DPO
RLHF: Reinforcement learning from human feedback: train a reward model on preferences, then optimise the model against it with reinforcement learning.; Explained in Fine-tuning methods: SFT, LoRA, QLoRA and DPO
ROUGE and BLEU: Metrics that compare word overlap between an output and a reference. They are weak for open-ended text, where many different answers are good.; Explained in Evaluating LLM systems
RRFAbbreviation: Short for reciprocal rank fusion: merging ranked lists by summing 1/(k + rank) per list.; See Reciprocal rank fusion (RRF)Explained in Better retrieval Coding, project defense and mock rounds

S

Sampling: Choosing the next token from the model's probability distribution rather than always taking the top one. Settings such as temperature, top-k and top-p shape it.; Explained in Sampling parameters and the model landscape
Self-attention: A step where each token looks at the visible tokens, including itself, and takes a weighted mix of their information. It lets the model link distant words and runs in parallel across positions.; Explained in How LLMs work
Self-consistency: Sampling several chain-of-thought answers and taking the majority. It raises reliability on short answers that can be voted on, at about N times the cost and latency.; Explained in Prompting fundamentals
Self-preference bias: A judge's tendency to favour outputs from its own model family. Using a judge from another family helps.; Explained in RAG evaluation in depth
Self-RAG: A RAG variant in which a specially trained model emits reflection tokens that decide whether to retrieve and judge whether passages are relevant and its answer is supported. It cannot be added to an ordinary model with a prompt.; Explained in All RAG types
Semantic cache: A cache that returns a stored answer for a similar question. It can serve a wrong answer or cross a tenant boundary, so use a strict threshold and scope keys.; Explained in Cost and latency optimization
Semantic memory: Stored facts about a user or domain, kept in a profile or key-value store.; Explained in Context engineering and memory
Semantic search: Finding items by meaning, using the closeness of embeddings, instead of by exact words. It can miss exact identifiers and rare terms.; Explained in Embeddings and similarity search
Sentence-window retrieval: Matching a single sentence and returning it with a few neighbouring sentences on each side.; Explained in Better retrieval
SFTAbbreviation: Short for supervised fine-tuning: fine-tuning on pairs of a prompt and an ideal response.; See Supervised fine-tuning (SFT)Explained in Fine-tuning methods: SFT, LoRA, QLoRA and DPO
Shadow traffic: Sending a copy of real requests to a new version without showing its answers to users, to compare behaviour safely.; Explained in RAG security, reliability and release Designing content moderation and guardrail systems
Softmax: A function that turns a list of scores into probabilities that sum to 1. Used on logits for the next token and inside attention.; Explained in How LLMs work
Speculative decoding: A cheap drafter, often a small model, proposes several tokens and the large model verifies them in one pass. The output distribution is preserved, and the speed-up depends on how often drafts are accepted.; Explained in LLM inference and serving
stdio transport: The MCP transport for local servers run as subprocesses of the host. Remote servers use Streamable HTTP instead.; Explained in Model Context Protocol and tool ecosystems
Stopping controls: The limits that end an agent loop: a checkable done criterion, step cap, time and cost budgets, loop detection and escalation to a human.; Explained in Agentic patterns: ReAct, planning and memory
Streamable HTTP: The MCP transport for remote servers, in which each message is an HTTP POST to one endpoint. It replaces the older, deprecated HTTP+SSE transport.; Explained in Model Context Protocol and tool ecosystems
Streaming: Sending output tokens as they are generated. It improves perceived latency, not total latency or cost.; Explained in Cost and latency optimization
Strict mode: A tool-calling setting that makes arguments follow the schema more reliably. It does not check permissions or whether the values make sense.; Explained in Function and tool calling
Structured output: A reply that follows a defined shape, such as JSON matching a schema. When the provider enforces the schema, the form is reliable but the values can still be wrong, and a truncated reply or a refusal may not match, so validate anyway.; Explained in Reasoning models and structured outputs
Supervised fine-tuning (SFT): Fine-tuning on pairs of a prompt and an ideal response.; Explained in Fine-tuning methods: SFT, LoRA, QLoRA and DPO
System prompt: Instructions set by the application that define the model's role and rules for a conversation. It helps steer behaviour but is not a security boundary.; Explained in Prompting fundamentals Guardrails, safety and prompt injection Limiting blast radius and treating output as untrusted

T

Temperature: A sampling setting that divides the logits before softmax. Higher values flatten the distribution (more random); lower values sharpen it. At 0 the choice is greedy, which is more repeatable but not guaranteed identical across runs.; Explained in Sampling parameters and the model landscape
Tensor parallelism: Splitting each layer's weight matrices across several GPUs, so every GPU holds a slice of every layer and they exchange partial results at each layer. It lets a model too large for one GPU run, and it needs fast links between the GPUs.; Explained in LLM inference and serving
Test-time compute: Extra computation spent when answering, rather than in training, to get better results. Longer reasoning, sampling several answers and verifying are examples.; Explained in Reasoning models and structured outputs
Throughput: The total work a system completes per unit time, such as tokens or requests per second.; Explained in LLM inference and serving
Time to first token (TTFT): The delay from sending a request until the first output token arrives. It is driven by queueing and prefill.; Explained in Cost and latency optimization LLM inference and serving
Token: The unit a model reads and writes: a word, part of a word, a symbol or a byte sequence. Cost, speed and context limits are counted in tokens, not words.; Explained in Tokenization and embeddings
Tokenizer: The component that splits text into tokens and maps them to integer ids, and back. Each model family has its own, so count tokens with the tokenizer of the model you call.; Explained in Tokenization and embeddings
Tombstone: A marker recording that a document was deleted, so the deletion reaches the index and the content stops being retrieved.; Explained in Production deployment of RAG
Tool definition: The name, description and parameter schema that tell the model what a tool does. It is all the model knows about the tool, so vague ones lead to wrong calls.; Explained in Function and tool calling
Tool-output bloat: Tool results that fill the context with more tokens than the step needs. Trim or summarise them before they re-enter the prompt.; Explained in Context engineering and memory
Top-k: A sampling filter that keeps only the k highest-probability tokens and samples among them. The cutoff is a fixed count, whatever the distribution looks like.; Explained in Sampling parameters and the model landscape
Top-p (nucleus sampling): A sampling filter that keeps the smallest set of tokens whose cumulative probability reaches p. It keeps few tokens when the model is confident and more when it is unsure.; Explained in Sampling parameters and the model landscape
Trace: The recorded path of one request through a system, with each stage's inputs, outputs, timing and cost. Traces hold user data and need privacy controls.; Explained in Observability and eval pipelines
Trajectory evaluation: Judging the steps an agent took, such as tool choices, cost and risk, in addition to its final result.; Explained in Agent evaluation: state, trajectory, pass^k and release gates
Transformer: The neural network design behind modern LLMs. Stacked blocks of self-attention and feedforward layers process all positions of the input in parallel during training and prefill.; Explained in How LLMs work
TTFTAbbreviation: Short for time to first token: the delay from sending a request until the first output token arrives.; See Time to first token (TTFT)Explained in Cost and latency optimization LLM inference and serving

V

Validate, retry, fall back: The loop around model output: check it, retry a bounded number of times with the error message, then use a safe fallback. Also handle truncation and refusals.; Explained in Reasoning models and structured outputs
Vector database: A store built to hold embeddings and find the nearest ones quickly, usually with metadata filters. Start with what you already run and move when a measured limit forces it.; Explained in Vector databases
Vector normalization: Scaling a vector to length 1. After it, cosine equals the dot product, which is cheaper to compute.; Explained in Embeddings and similarity search
Verbosity bias: A judge's tendency to prefer longer answers regardless of quality.; Explained in RAG evaluation in depth
Vision-language model (VLM): A model that takes images as well as text. A vision encoder turns patches of the image into tokens the language model can read, so more images cost more.; Explained in Multimodal GenAI and document AI
VLMAbbreviation: Short for vision-language model: a model that takes images as well as text.; See Vision-language model (VLM)Explained in Multimodal GenAI and document AI
Vocabulary: The fixed set of tokens a model can read and produce. A bigger vocabulary gives fewer tokens per text but a larger embedding table and output layer.; Explained in Tokenization and embeddings

W

Workflow: An LLM system that follows a path predefined in code. Prefer it when the path is predictable.; Explained in Agentic patterns: ReAct, planning and memory
Working memory: In the course's four-way split of agent memory, whatever is in the context window or a rolling summary.; Explained in Context engineering and memory

Z

Zero-shot prompting: Asking the model to do a task with instructions only and no worked examples. It is the cheapest starting point.; Explained in Prompting fundamentals