Results for "preference optimization"

A preference-based training method optimizing policies directly from pairwise comparisons without explicit RL loops.

Reinforcement learning from human feedback: uses preference data to train a reward model and optimize the policy.

Inferring reward function from observed behavior.

Inferring and aligning with human preferences.

Optimization with multiple local minima/saddle points; typical in neural networks.

Optimization problems where any local minimum is global.

Optimization under equality/inequality constraints.

Optimizing continuous action sequences.

A point where gradient is zero but is neither a max nor min; common in deep nets.

Optimization using curvature information; often expensive at scale.

Visualization of optimization landscape.

Restricting updates to safe regions.

Methods like Adam adjusting learning rates dynamically.

A function measuring prediction error (and sometimes calibration), guiding gradient-based optimization.

The shape of the loss function over parameter space.

Adjusting learning rate over training to improve convergence.

Distributed agents producing emergent intelligence.

Minimum relative to nearby points.

Flat high-dimensional regions slowing training.

Choosing step size along gradient direction.

Converts constrained problem to unconstrained form.

Alternative formulation providing bounds.

Fast approximation of costly simulations.

Configuration choices not learned directly (or not typically learned) that govern training or architecture.

Uses an exponential moving average of gradients to speed convergence and reduce oscillation.

Controls the size of parameter updates; too high diverges, too low trains slowly or gets stuck.

Variability introduced by minibatch sampling during SGD.

Limiting gradient magnitude to prevent exploding gradients.

Matrix of second derivatives describing local curvature of loss.

Matrix of curvature information.