Bias

Techniques to understand model decisions (global or local), important in high-stakes and regulated settings.

Studying internal mechanisms or input influence on outputs (e.g., saliency maps, SHAP, attention analysis).

Feature attribution method grounded in cooperative game theory for explaining predictions in tabular settings.

Local surrogate explanation method approximating model behavior near a specific input.

Fine-tuning on (prompt, response) pairs to align a model with instruction-following behaviors.

Model trained to predict human preferences (or utility) for candidate outputs; used in RLHF-style pipelines.

Ensuring model behavior matches human goals, norms, and constraints, including reducing harmful or deceptive outputs.

Automated detection/prevention of disallowed outputs (toxicity, self-harm, illegal instruction, etc.).