Prompt caching

A provider feature that reuses the computed state of an identical prompt prefix. Cache reads cost a fraction of normal input and the first token comes sooner, while cache writes can cost a premium. Put static content first. Not the same as prefix caching, which happens inside your own serving engine.

All terms, A to Z