Autoregressive generation

Producing text one token at a time, where each new token is appended to the input before the next step. This is why output length drives latency.

All terms, A to Z