Vanishing gradient problem
Problema de desvanecimiento de gradiente
- Date
- 1991 · confidence high
- Place
- Munich, Bavaria, Germany
- Credited to
- Sepp Hochreiter
- Field
- Knowledge & symbols
What was discovered
Deep learning problem in which gradients become extremely small as they backpropagate through many layers, preventing effective training of deep neural networks; identified by Sepp Hochreiter in his 1991 thesis.
What it led to
- long short-term memory (LSTM) networks
- ReLU activation functions
- residual network architectures (ResNet)
- advances in modern deep learning