Results for "generalization"

How well a model performs on new data drawn from the same (or similar) distribution as training.

A narrow minimum often associated with poorer generalization.

Number of samples per gradient update; impacts compute efficiency, generalization, and stability.

A wide basin often correlated with better generalization.

Training one model on multiple tasks simultaneously to improve generalization through shared structure.

Ordering training samples from easier to harder to improve convergence or generalization.

Built-in assumptions guiding learning efficiency and generalization.

A measure of a model class’s expressive capacity based on its ability to shatter datasets.

Measures a model’s ability to fit random noise; used to bound generalization error.

Minimizing average loss on training data; can overfit when data is limited or biased.

Techniques that discourage overly complex solutions to improve generalization (reduce overfitting).

Randomly zeroing activations during training to reduce co-adaptation and overfitting.

When information from evaluation data improperly influences training, inflating reported performance.

The set of tokens a model can represent; impacts efficiency, multilinguality, and handling of rare strings.

A high-capacity language model trained on massive corpora, exhibiting broad generalization and emergent behaviors.

Expanding training data via transformations (flips, noise, paraphrases) to improve robustness.

Artificially created data used to train/test models; helpful for privacy and coverage, risky if unrealistic.

Automated testing and deployment processes for models and data workflows, extending DevOps to ML artifacts.

A theoretical framework analyzing what classes of functions can be learned, how efficiently, and with what guarantees.

Error due to sensitivity to fluctuations in the training dataset.

Using same parameters across different parts of a model.

The range of functions a model can represent.

Encodes positional information via rotation in embedding space.

Increasing performance via more data.

Measure of spread around the mean.

Ensuring learned behavior matches intended objective.

Applying learned patterns incorrectly.

Loss of old knowledge when learning new tasks.

Train/test environment mismatch.

Randomizing simulation parameters to improve real-world transfer.