Results for "relative entropy"

A measure of randomness or uncertainty in a probability distribution.

Measures divergence between true and predicted probability distributions.

Reduction in uncertainty achieved by observing a variable; used in decision trees and active learning.

Encodes positional information via rotation in embedding space.

Accelerating safety relative to capabilities.

Quantifies shared information between random variables.

Attention where queries/keys/values come from the same sequence, enabling token-to-token interactions.

Injects sequence order into Transformers, since attention alone is permutation-invariant.

Raw model outputs before converting to probabilities; manipulated during decoding and calibration.

Measures how one probability distribution diverges from another.

Minimum relative to nearby points.

Mathematical representation of friction forces.

Estimating robot position within a map.

Returns above benchmark.

A scalar measure optimized during training, typically expected loss over data, sometimes with regularization terms.

A function measuring prediction error (and sometimes calibration), guiding gradient-based optimization.

Penalizes confident wrong predictions heavily; standard for classification and language modeling.

Training objective where the model predicts the next token given previous tokens (causal modeling).

Predicts masked tokens in a sequence, enabling bidirectional context; often used for embeddings rather than generation.

Model that compresses input into latent space and reconstructs it.

Assigning labels per pixel (semantic) or per instance (instance segmentation) to map object boundaries.

Assigning category labels to images.

Pixel-wise classification of image regions.

Learning policies from expert demonstrations.

Learning action mapping directly from demonstrations.