Results for "shortcut learning"

Converting text into discrete units (tokens) for modeling; subword tokenizers balance vocabulary size and coverage.

Networks with recurrent connections for sequences; largely supplanted by Transformers for many tasks.

A datastore optimized for similarity search over embeddings, enabling semantic retrieval at scale.

An RNN variant using gates to mitigate vanishing gradients and capture longer context.

Fine-tuning on (prompt, response) pairs to align a model with instruction-following behaviors.

Architecture based on self-attention and feedforward layers; foundation of modern LLMs and many multimodal models.

Ensuring model behavior matches human goals, norms, and constraints, including reducing harmful or deceptive outputs.

A high-capacity language model trained on massive corpora, exhibiting broad generalization and emergent behaviors.

Automated detection/prevention of disallowed outputs (toxicity, self-harm, illegal instruction, etc.).

Techniques to understand model decisions (global or local), important in high-stakes and regulated settings.

Studying internal mechanisms or input influence on outputs (e.g., saliency maps, SHAP, attention analysis).

Local surrogate explanation method approximating model behavior near a specific input.

Framework for reasoning about cause-effect relationships beyond correlation, often using structural assumptions and experiments.

A hidden variable influences both cause and effect, biasing naive estimates of causal impact.

Protecting data during network transfer and while stored; essential for ML pipelines handling sensitive data.

Artificially created data used to train/test models; helpful for privacy and coverage, risky if unrealistic.

Processes and controls for data quality, access, lineage, retention, and compliance across the AI lifecycle.

Systematic review of model/data processes to ensure performance, fairness, security, and policy compliance.

Central system to store model versions, metadata, approvals, and deployment state.

Logging hyperparameters, code versions, data snapshots, and results to reproduce and compare experiments.

Ability to replicate results given same code/data; harder in distributed training and nondeterministic ops.

Time from request to response; critical for real-time inference and UX.

How many requests or tokens can be processed per unit time; affects scalability and cost.

Hardware resources used for training/inference; constrained by memory bandwidth, FLOPs, and parallelism.

Reducing numeric precision of weights/activations to speed inference and reduce memory with acceptable accuracy loss.

Removing weights or neurons to shrink models and improve efficiency; can be structured or unstructured.

Converts logits to probabilities by exponentiation and normalization; common in classification and LMs.

A dataset + metric suite for comparing models; can be gamed or misaligned with real-world goals.

System for running consistent evaluations across tasks, versions, prompts, and model settings.

Stress-testing models for failures, vulnerabilities, policy violations, and harmful behaviors before release.