Reverse-mode automatic differentiation (mathematical origin of backpropagation)
Diferenciación automática en modo inverso (origen matemático de la retropropagación)
- Date
- 1970 · confidence medium
- Place
- University of Helsinki, Helsinki, Finland
- Credited to
- Seppo Linnainmaa
- Field
- Knowledge & symbols
What was discovered
In his master's thesis in Helsinki, Linnainmaa formalized reverse accumulation (reverse-mode automatic differentiation): applying the chain rule from output back to inputs to compute derivatives efficiently. He published the result in a journal in 1976.
The catalog already includes the application of backpropagation to neural network training (1986, Rumelhart, Hinton and Williams); this entry records the earlier mathematical origin of the method, Linnainmaa's reverse-mode automatic differentiation. Frank Rosenblatt had already used the term "back-propagating error correction" in 1962 without managing to implement it.
What it led to
- Mathematical basis of backpropagation used to train neural networks
- Modern automatic differentiation tools in deep learning
- Efficient optimization of functions with many variables