9 citations · 9 across the 2 of their papers we have counts for
4 papers · 1 filter
An Empirical Study of the Occurrence of Heavy-Tails in Training a ReLU Gate
Sayar Karmakar, Anirbit Mukherjee
A particular direction of recent advance about stochastic deep-learning algorithms has been about uncovering a rather mysterious heavy-tailed nature of the stationary distribution…
A Study of the Mathematics of Deep Learning
Anirbit Mukherjee
"Deep Learning"/"Deep Neural Nets" is a technological marvel that is now increasingly deployed at the cutting-edge of artificial intelligence tasks. This dramatic success of deep l…
Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov acceleration
Soham De, Anirbit Mukherjee, Enayat Ullah
RMSProp and ADAM continue to be extremely popular algorithms for training neural nets but their theoretical convergence properties have remained unclear. Further, recent work has s…
Sparse Coding and Autoencoders
Akshay Rangamani, Anirbit Mukherjee, Amitabh Basu +4
In "Dictionary Learning" one tries to recover incoherent matrices (typically overcomplete and whose columns are assumed to be normalized) and spar…