Constrained Optimization

Converts constrained problem to unconstrained form.

Alternative formulation providing bounds.

Optimization under uncertainty.

Methods like Adam adjusting learning rates dynamically.

Lowest possible loss.

Flat high-dimensional regions slowing training.

Choosing step size along gradient direction.

Restricting updates to safe regions.