6 papers · 1 filter
Finite-Time Concentration and Convergence Rates for Projected Two-Time-Scale Stochastic Approximation with Markov Noise
Rahul Singh, Vivek S. Borkar, Eric Moulines
We study finite-time concentration and convergence rates for projected two-time-scale stochastic approximation driven by a controlled Markov chain. The averaged fast map is contrac…
Stochastic gradient descent with discontinuity across a manifold
Vivek S. Borkar
Stochastic gradient descent for a loss function discontinuous across lower dimensional manifolds is analyzed by studying its differential equation limit.
Exploding and vanishing gradients in deep neural networks: the effect of residual connections
Vivek S Borkar
The well known phenomenon of exploding and vanishing gradients in deep neural networks is analyzed using multiplicative ergodic theory. The effect of adding a residual connection i…
Adynamical systems view of training generativemodels and the memorization phenomenon
Siva Athreya, Chiranjib Bhattacharya, Vivek S. Borkar
Using recent works of one of the authors (VSB) on collapse in generative models and two time scale dynamics in stochastic gradient descent in high dimensions, we give a system theo…
Stochastic approximation in non-markovian environments revisited
Vivek Shripad Borkar
Based on some recent work of the author on stochastic approximation in non-markovian environments, the situation when the driving random process is non-ergodic in addition to being…
Small noise asymptotics for a class of jump-diffusions with heavy tails for large times
Sumith Reddy Anugu, Siva R. Athreya, Vivek S. Borkar
In this work, we investigate positive recurrent Lévy diffusions driven by appropriately scaled Brownian motion and -stable process (with ) in the small noise regime. Supp…