Glossary
Prefill/decode disaggregation
Running prefill and decode on separate GPU pools so the two phases do not interfere. It costs more machines and KV transfer.
Where it is explained
See also
Terms explained in the same lessons.
Glossary
Running prefill and decode on separate GPU pools so the two phases do not interfere. It costs more machines and KV transfer.
Terms explained in the same lessons.