MAP Estimation

Optimization problems where any local minimum is global.

Optimization with multiple local minima/saddle points; typical in neural networks.

A point where gradient is zero but is neither a max nor min; common in deep nets.

Variability introduced by minibatch sampling during SGD.

Quantifies shared information between random variables.

Measures how much information an observable random variable carries about unknown parameters.

Estimating parameters by maximizing likelihood of observed data.

Updating beliefs about parameters using observed evidence and prior distributions.