RLHF

Intermediate

Reinforcement learning from human feedback: uses preference data to train a reward model and optimize the policy.

Full Definition

Reinforcement learning from human feedback: uses preference data to train a reward model and optimize the policy.

Keywords

preference optimization

Domains

Optimization

Related Terms

Reward Model used in

Model trained to predict human preferences (or utility) for candidate outputs; used in RLHF-style pipelines.

 Alignment related to

Ensuring model behavior matches human goals, norms, and constraints, including reducing harmful or deceptive outputs.

 DPO related to

A preference-based training method optimizing policies directly from pairwise comparisons without explicit RL loops.

 DPO related to

A preference-based training method optimizing policies directly from pairwise comparisons without explicit RL loops.

 Mean Squared Error related to

Average of squared residuals; common regression objective.

 Gradient Descent related to

Iterative method that updates parameters in the direction of negative gradient to minimize loss.

 Momentum related to

Uses an exponential moving average of gradients to speed convergence and reduce oscillation.

 Adam related to

Popular optimizer combining momentum and per-parameter adaptive step sizes via first/second moment estimates.

Concept Map

See how RLHF connects to other concepts.

Open Knowledge Graph