Results for "optimization duality"

Alternative formulation providing bounds.

Optimization with multiple local minima/saddle points; typical in neural networks.

Optimization problems where any local minimum is global.

Optimization under equality/inequality constraints.

Optimizing continuous action sequences.

A point where gradient is zero but is neither a max nor min; common in deep nets.

Optimization using curvature information; often expensive at scale.

Visualization of optimization landscape.

Restricting updates to safe regions.

Methods like Adam adjusting learning rates dynamically.

A function measuring prediction error (and sometimes calibration), guiding gradient-based optimization.

The shape of the loss function over parameter space.

Adjusting learning rate over training to improve convergence.

Minimum relative to nearby points.

Distributed agents producing emergent intelligence.

Flat high-dimensional regions slowing training.

Choosing step size along gradient direction.

Converts constrained problem to unconstrained form.

Fast approximation of costly simulations.

Configuration choices not learned directly (or not typically learned) that govern training or architecture.

Uses an exponential moving average of gradients to speed convergence and reduce oscillation.

Controls the size of parameter updates; too high diverges, too low trains slowly or gets stuck.

Variability introduced by minibatch sampling during SGD.

Limiting gradient magnitude to prevent exploding gradients.

Matrix of second derivatives describing local curvature of loss.

Matrix of curvature information.

Optimizing policies directly via gradient ascent on expected reward.

Optimization under uncertainty.

Measure of vector magnitude; used in regularization and optimization.

Model optimizes objectives misaligned with human values.