Glossary
Grouped-query attention (GQA)
An attention variant in which several query heads share one set of key and value heads. Fewer KV heads means a smaller KV cache. Multi-query attention (MQA) is the extreme case with one KV head.
- Also written as: GQA
Where it is explained
See also
Terms explained in the same lessons.