Glossary
Prefill
The first phase of a request, in which the whole prompt is processed in parallel. It sets time to first token and is usually limited by compute.
Where it is explained
See also
Terms explained in the same lessons.
Glossary
The first phase of a request, in which the whole prompt is processed in parallel. It sets time to first token and is usually limited by compute.
Terms explained in the same lessons.