Saddle Plateau

Intermediate

Flat high-dimensional regions slowing training.

Full Definition

Flat high-dimensional regions slowing training.

Keywords

flat region

Domains

Foundations & Theory

Related Terms

Line Search related to

Choosing step size along gradient direction.

 Trust Region related to

Restricting updates to safe regions.

 Constrained Optimization related to

Optimization under equality/inequality constraints.

 Lagrangian related to

Converts constrained problem to unconstrained form.

 NLP related to

AI subfield dealing with understanding and generating human language, including syntax, semantics, and pragmatics.

 Objective Surface related to

Visualization of optimization landscape.

 Local Minimum related to

Minimum relative to nearby points.

 Global Minimum related to

Lowest possible loss.

Concept Map

See how Saddle Plateau connects to other concepts.

Open Knowledge Graph