Results for "ReLU"

Nonlinear functions enabling networks to approximate complex mappings; ReLU variants dominate modern DL.

Gradients shrink through layers, slowing learning in early layers; mitigated by ReLU, residuals, normalization.

Activation max(0, x); improves gradient flow and training speed in deep nets.