Top-k

Samples from the smallest set of tokens whose probabilities sum to p, adapting set size by context.

Raw model outputs before converting to probabilities; manipulated during decoding and calibration.

Converts logits to probabilities by exponentiation and normalization; common in classification and LMs.

System for running consistent evaluations across tasks, versions, prompts, and model settings.

Training a smaller “student” model to mimic a larger “teacher,” often improving efficiency while retaining performance.

Search algorithm for generation that keeps top-k partial sequences; can improve likelihood but reduce diversity.

Stochastic generation strategies that trade determinism for diversity; key knobs include temperature and nucleus samp...

Scales logits before sampling; higher increases randomness/diversity, lower increases determinism.