Reinforced stochastic gradient descent for deep neural network learning
arXiv:1701.07974
Abstract
Stochastic gradient descent (SGD) is a standard optimization method to minimize a training error with respect to network parameters in modern neural network learning. However, it typically suffers from proliferation of saddle points in the high-dimensional parameter space. Therefore, it is highly desirable to design an efficient algorithm to escape from these saddle points and reach a parameter region of better generalization capabilities. Here, we propose a simple extension of SGD, namely reinforced SGD, which simply adds previous first-order gradients in a stochastic manner with a probability that increases with learning time. As verified in a simple synthetic dataset, this method significantly accelerates learning compared with the original SGD. Surprisingly, it dramatically reduces over-fitting effects, even compared with state-of-the-art adaptive learning algorithm---Adam. For a benchmark handwritten digits dataset, the learning performance is comparable to Adam, yet with an extra advantage of requiring one-fold less computer memory. The reinforced SGD is also compared with SGD with fixed or adaptive momentum parameter and Nesterov's momentum, which shows that the proposed framework is able to reach a similar generalization accuracy with less computational costs. Overall, our method introduces stochastic memory into gradients, which plays an important role in understanding how gradient-based training algorithms can work and its relationship with generalization abilities of deep networks.
arXiv comment12 pages and 9 figures, nearly final version as a technical report
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- On the difficulty of training Recurrent Neural Networks
- The Marginal Value of Adaptive Gradient Methods in Machine Learning
- Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
- Adding Gradient Noise Improves Learning for Very Deep Networks
- Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition
- Unreasonable Effectiveness of Learning Neural Networks: From Accessible States and Robust Ensembles to Basic Algorithmic Schemes
- The statistics of critical points of Gaussian fields on large-dimensional spaces
- Learning by message-passing in networks of discrete synapses
- Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses
- Entropy-SGD: Biasing Gradient Descent Into Wide Valleys
- Origin of the computational hardness for learning with binary synapses