Glossary
PagedAttention
Storing the KV cache in on-demand blocks to avoid fragmentation, which allows larger batches and prefix sharing.
Where it is explained
See also
Terms explained in the same lessons.
Glossary
Storing the KV cache in on-demand blocks to avoid fragmentation, which allows larger batches and prefix sharing.
Terms explained in the same lessons.