Context window

The maximum number of tokens a model can take in and produce in one request. A larger window is not free: you pay per token, and attention work and KV-cache memory grow with length.

All terms, A to Z