1 citations · 1 across the 6 of their papers we have counts for
4 papers · 1 filter
High-Probability Convergence of SGD via Batched Updates
Feng Zhu, Robert W. Heath, Aritra Mitra
Stochastic gradient descent (SGD) is the primary workhorse for large-scale optimization. While the average behavior of its iterates, typically characterized by mean-squared error b…
Variance-Reduced Q-Learning over Static and Time-Varying Networks
Sreejeet Maity, Feng Zhu, Aritra Mitra +1
We investigate a decentralized reinforcement learning problem involving multiple agents that interact with the same Markov Decision Process (MDP). The agents can exchange informati…
Lifelong In-Context Learning with Transformers Requires Parametric Forms of Attention
Luke McDermott, Robert W. Heath, Rahul Parhi
Lifelong continual learning remains an obstacle on the path to human-like intelligence. Modern transformers show sparks of intelligence with in-context learning. The quadratic natu…
A Short and Unified Convergence Analysis of the SAG, SAGA, and IAG Algorithms
Feng Zhu, Robert W. Heath, Aritra Mitra
Stochastic variance-reduced algorithms such as Stochastic Average Gradient (SAG) and SAGA, and their deterministic counterparts like the Incremental Aggregated Gradient (IAG) metho…