Difficulty: Intermediate
Systematic review of model/data processes to ensure performance, fairness, security, and policy compliance.
Logged record of model inputs, outputs, and decisions.
Ability to inspect and verify AI decisions.
Tendency to trust automated suggestions even when incorrect; mitigated by UI design, training, and checks.
Generates sequences one token at a time, conditioning on past tokens.
Dynamic resource allocation.
Hidden behavior activated by specific triggers, causing targeted mispredictions or undesired outputs.
Running predictions on large datasets periodically.
Number of samples per gradient update; impacts compute efficiency, generalization, and stability.
Updating beliefs about parameters using observed evidence and prior distributions.
Search algorithm for generation that keeps top-k partial sequences; can improve likelihood but reduce diversity.
Fundamental recursive relationship defining optimal value functions.
A dataset + metric suite for comparing models; can be gamed or misaligned with real-world goals.
Systematic differences in model outcomes across groups; arises from data, labels, and deployment context.
Systematic error introduced by simplifying assumptions in a learning algorithm.
A conceptual framework describing error as the sum of systematic error (bias) and sensitivity to data (variance).
Maintaining two environments for instant rollback.
Probabilistic energy-based neural network with hidden variables.
A narrow hidden layer forcing compact representations.
A proper scoring rule measuring squared error of predicted probabilities for binary outcomes.
Storing results to reduce compute.
The degree to which predicted probabilities match true frequencies (e.g., 0.8 means ~80% correct).
Incrementally deploying new models to reduce risk.
Detecting unauthorized model outputs or data leaks.
Predicting case success probabilities.
Loss of old knowledge when learning new tasks.
Framework for reasoning about cause-effect relationships beyond correlation, often using structural assumptions and experiments.
Prevents attention to future tokens during training/inference.
Stepwise reasoning patterns that can improve multi-step tasks; often handled implicitly or summarized for safety/privacy.
Governance of model changes.