Glossary
Latency budget
An end-to-end time target split among stages such as retrieval, rerank and generation, so each stage has a limit.
Where it is explained
See also
Terms explained in the same lessons.
Glossary
An end-to-end time target split among stages such as retrieval, rerank and generation, so each stage has a limit.
Terms explained in the same lessons.