Bottleneck Layer

Using same parameters across different parts of a model.

Built-in assumptions guiding learning efficiency and generalization.

The range of functions a model can represent.

Tradeoffs between many layers vs many neurons per layer.

Matrix of second derivatives describing local curvature of loss.

Neural networks can approximate any continuous function under certain conditions.

Allows gradients to bypass layers, enabling very deep networks.

Early architecture using learned gates for skip connections.