Results for "trial-and-error"

When a model cannot capture underlying structure, performing poorly on both training and test data.

Harmonic mean of precision and recall; useful when balancing false positives/negatives matters.

Penalizes confident wrong predictions heavily; standard for classification and language modeling.

Uses an exponential moving average of gradients to speed convergence and reduce oscillation.

Popular optimizer combining momentum and per-parameter adaptive step sizes via first/second moment estimates.

Number of samples per gradient update; impacts compute efficiency, generalization, and stability.

Activation max(0, x); improves gradient flow and training speed in deep nets.

Techniques that stabilize and speed training by normalizing activations; LayerNorm is common in Transformers.

Randomly zeroing activations during training to reduce co-adaptation and overfitting.

An RNN variant using gates to mitigate vanishing gradients and capture longer context.

Mechanism that computes context-aware mixtures of representations; scales well and captures long-range dependencies.

Converting text into discrete units (tokens) for modeling; subword tokenizers balance vocabulary size and coverage.

The set of tokens a model can represent; impacts efficiency, multilinguality, and handling of rare strings.

A high-capacity language model trained on massive corpora, exhibiting broad generalization and emergent behaviors.

The text (and possibly other modalities) given to an LLM to condition its output behavior.

Crafting prompts to elicit desired behavior, often using role, structure, constraints, and examples.

Architecture that retrieves relevant documents (e.g., from a vector DB) and conditions generation on them to reduce hallucinations.

Retrieval based on embedding similarity rather than keyword overlap, capturing paraphrases and related concepts.

Model-generated content that is fluent but unsupported by evidence or incorrect; mitigated by grounding and verification.

Reinforcement learning from human feedback: uses preference data to train a reward model and optimize the policy.

Ensuring model behavior matches human goals, norms, and constraints, including reducing harmful or deceptive outputs.

Rules and controls around generation (filters, validators, structured outputs) to reduce unsafe or invalid behavior.

Systematic differences in model outcomes across groups; arises from data, labels, and deployment context.

Techniques to understand model decisions (global or local), important in high-stakes and regulated settings.

Framework for reasoning about cause-effect relationships beyond correlation, often using structural assumptions and experiments.

A hidden variable influences both cause and effect, biasing naive estimates of causal impact.

Human or automated process of assigning targets; quality, consistency, and guidelines matter heavily.

Artificially created data used to train/test models; helpful for privacy and coverage, risky if unrealistic.

Protecting data during network transfer and while stored; essential for ML pipelines handling sensitive data.

Information that can identify an individual (directly or indirectly); requires careful handling and compliance.