Key-Value Cache

Intermediate

Stores past attention states to speed up autoregressive decoding.

Full Definition

Stores past attention states to speed up autoregressive decoding.

Keywords

Domains

Related Terms

Concept Map

See how Key-Value Cache connects to other concepts.

Open Knowledge Graph