Fast, Better Training Trick -- Random Gradient
arXiv:1808.04293
Abstract
In this paper, we will show an unprecedented method to accelerate training and improve performance, which called random gradient (RG). This method can be easier to the training of any model without extra calculation cost, we use Image classification, Semantic segmentation, and GANs to confirm this method can improve speed which is training model in computer vision. The central idea is using the loss multiplied by a random number to random reduce the back-propagation gradient. We can use this method to produce a better result in Pascal VOC, Cifar, Cityscapes datasets.
arXiv commentarXiv admin note: text overlap with arXiv:1708.07120 by other authors
References in corpus (10)
- Adam: A Method for Stochastic Optimization
- Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
- The Cityscapes Dataset for Semantic Urban Scene Understanding
- Don't Decay the Learning Rate, Increase the Batch Size
- Large Batch Training of Convolutional Networks
- Three Factors Influencing Minima in SGD
- Entropy-SGD: Biasing Gradient Descent Into Wide Valleys
- Adaptive Subgradient Methods for Online AUC Maximization