1.9k citations · 3.4k across the 51 of their papers we have counts for
5 papers · 1 filter
Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark
Shiv Ram Dubey, Satish Kumar Singh, Bidyut Baran Chaudhuri
Neural networks have shown tremendous growth in recent years to solve numerous problems. Various types of neural networks have been introduced to deal with different types of probl…
AdaInject: Injection Based Adaptive Gradient Descent Optimizers for Convolutional Neural Networks
Shiv Ram Dubey, S. H. Shabbeer Basha, Satish Kumar Singh +1
The convolutional neural networks (CNNs) are generally trained using stochastic gradient descent (SGD) based optimization techniques. The existing SGD optimizers generally suffer w…
AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networks
S. K. Roy, M. E. Paoletti, J. M. Haut +4
Convolutional neural networks (CNNs) are trained using stochastic gradient descent (SGD)-based optimizers. Recently, the adaptive moment estimation (Adam) optimizer has become very…
NASIB: Neural Architecture Search withIn Budget
Abhishek Singh, Anubhav Garg, Jinan Zhou +2
Neural Architecture Search (NAS) represents a class of methods to generate the optimal neural network architecture and typically iterate over candidate architectures till convergen…
diffGrad: An Optimization Method for Convolutional Neural Networks
Shiv Ram Dubey, Soumendu Chakraborty, Swalpa Kumar Roy +3
Stochastic Gradient Decent (SGD) is one of the core techniques behind the success of deep neural networks. The gradient provides information on the direction in which a function ha…