DPO

Intermediate

A preference-based training method optimizing policies directly from pairwise comparisons without explicit RL loops.

Full Definition

A preference-based training method optimizing policies directly from pairwise comparisons without explicit RL loops.

Keywords

direct preference optimization

Domains

Optimization

Related Terms

RLHF related to

Reinforcement learning from human feedback: uses preference data to train a reward model and optimize the policy.

 Gradient Descent related to

Iterative method that updates parameters in the direction of negative gradient to minimize loss.

 Momentum related to

Uses an exponential moving average of gradients to speed convergence and reduce oscillation.

 Adam related to

Popular optimizer combining momentum and per-parameter adaptive step sizes via first/second moment estimates.

 RLHF related to

Reinforcement learning from human feedback: uses preference data to train a reward model and optimize the policy.

Concept Map

See how DPO connects to other concepts.

Open Knowledge Graph