Results for "training loss"

Fabrication of cases or statutes by LLMs.

End-to-end process for model training.

Methods that learn training procedures or initializations so models can adapt quickly to new tasks with little data.

Techniques that discourage overly complex solutions to improve generalization (reduce overfitting).

How well a model performs on new data drawn from the same (or similar) distribution as training.

Controls the size of parameter updates; too high diverges, too low trains slowly or gets stuck.

Gradients shrink through layers, slowing learning in early layers; mitigated by ReLU, residuals, normalization.

Achieving task performance by providing a small number of examples inside the prompt without weight updates.

Fine-tuning on (prompt, response) pairs to align a model with instruction-following behaviors.

Model trained to predict human preferences (or utility) for candidate outputs; used in RLHF-style pipelines.

Reducing numeric precision of weights/activations to speed inference and reduce memory with acceptable accuracy loss.

Flat high-dimensional regions slowing training.

When information from evaluation data improperly influences training, inflating reported performance.

Training one model on multiple tasks simultaneously to improve generalization through shared structure.

Predicts masked tokens in a sequence, enabling bidirectional context; often used for embeddings rather than generation.

Autoencoder using probabilistic latent variables and KL regularization.

Hidden behavior activated by specific triggers, causing targeted mispredictions or undesired outputs.

Generator produces limited variety of outputs.

Assigning category labels to images.

Learning action mapping directly from demonstrations.

Learning a function from input-output pairs (labeled data), optimizing performance on predicting outputs for unseen inputs.

Reusing knowledge from a source task/domain to improve learning on a target task/domain, typically via pretrained models.

Learning where data arrives sequentially and the model updates continuously, often under changing distributions.

The degree to which predicted probabilities match true frequencies (e.g., 0.8 means ~80% correct).

Automatically learning useful internal features (latent variables) that capture salient structure for downstream tasks.

A parameterized function composed of interconnected units organized in layers with nonlinear activations.

Maximum number of tokens the model can attend to in one forward pass; constrains long-document reasoning.

Local surrogate explanation method approximating model behavior near a specific input.

Removing weights or neurons to shrink models and improve efficiency; can be structured or unstructured.

Optimization problems where any local minimum is global.