Results for "entropy reduction"

Reduction in uncertainty achieved by observing a variable; used in decision trees and active learning.

A measure of randomness or uncertainty in a probability distribution.

Measures divergence between true and predicted probability distributions.

Quantifies shared information between random variables.

Model that compresses input into latent space and reconstructs it.

Decomposes a matrix into orthogonal components; used in embeddings and compression.

A scalar measure optimized during training, typically expected loss over data, sometimes with regularization terms.

A function measuring prediction error (and sometimes calibration), guiding gradient-based optimization.

Penalizes confident wrong predictions heavily; standard for classification and language modeling.

Training objective where the model predicts the next token given previous tokens (causal modeling).

Predicts masked tokens in a sequence, enabling bidirectional context; often used for embeddings rather than generation.

Assigning category labels to images.

Assigning labels per pixel (semantic) or per instance (instance segmentation) to map object boundaries.

Pixel-wise classification of image regions.

Learning policies from expert demonstrations.

Learning action mapping directly from demonstrations.

Learning structure from unlabeled data, such as discovering groups, compressing representations, or modeling data distributions.

A measurable property or attribute used as model input (raw or engineered), such as age, pixel intensity, or token ID.

The internal space where learned representations live; operations here often correlate with semantics or generative factors.

A narrow hidden layer forcing compact representations.

All possible configurations an agent may encounter.

Vector whose direction remains unchanged under linear transformation.

Generative model that learns to reverse a gradual noise process.

Number of linearly independent rows or columns.

Approximating expectations via random sampling.

Sampling from easier distribution with reweighting.

Storing results to reduce compute.

Modeling environment evolution in latent space.

Risk threatening humanity’s survival.