184 citations · 526 across the 38 of their papers we have counts for
39 papers
Provable Length Generalization in Sequence Prediction via Spectral Filtering
Annie Marsden, Evan Dogariu, Naman Agarwal +3
We consider the problem of length generalization in sequence prediction. We define a new metric of performance in this setting -- the Asymmetric-Regret -- which measures regret aga…
FutureFill: Fast Generation from Convolutional Sequence Models
Naman Agarwal, Xinyi Chen, Evan Dogariu +6
We address the challenge of efficient auto-regressive generation in sequence prediction models by introducing FutureFill, a general-purpose fast generation method for any sequence…
Stacking as Accelerated Gradient Descent
Naman Agarwal, Pranjal Awasthi, Satyen Kale +1
Stacking, a heuristic technique for training deep residual networks by progressively increasing the number of layers and initializing new layers by copying parameters from older la…
Improved Differentially Private and Lazy Online Convex Optimization
Naman Agarwal, Satyen Kale, Karan Singh +1
We study the task of -differentially private online convex optimization (OCO). In the online setting, the release of each distinct decision or iterate carries with it the p…
Spectral State Space Models
Naman Agarwal, Daniel Suo, Xinyi Chen +1
This paper studies sequence modeling for prediction tasks with long range dependencies. We propose a new formulation for state space models (SSMs) based on learning linear dynamica…
HAVE-Net: Hallucinated Audio-Visual Embeddings for Few-Shot Classification with Unimodal Cues
Ankit Jha, Debabrata Pal, Mainak Singha +2
Recognition of remote sensing (RS) or aerial images is currently of great interest, and advancements in deep learning algorithms added flavor to it in recent years. Occlusion, intr…
Benchmarking Neural Network Training Algorithms
George E. Dahl, Frank Schneider, Zachary Nado +22
Training algorithms, broadly construed, are an essential part of every deep learning pipeline. Training algorithm improvements that speed up training across a wide variety of workl…
Variance-Reduced Conservative Policy Iteration
Naman Agarwal, Brian Bullins, Karan Singh
We study the sample complexity of reducing reinforcement learning to a sequence of empirical risk minimization problems over the policy space. Such reductions-based algorithms exhi…
Best of Both Worlds in Online Control: Competitive Ratio and Policy Regret
Gautam Goel, Naman Agarwal, Karan Singh +1
We consider the fundamental problem of online control of a linear dynamical system from two different viewpoints: regret minimization and competitive analysis. We prove that the op…
Multi-User Reinforcement Learning with Low Rank Rewards
Naman Agarwal, Prateek Jain, Suhas Kowshik +2
In this work, we consider the problem of collaborative multi-user reinforcement learning. In this setting there are multiple users with the same state-action space and transition p…
Adaptive Gradient Methods at the Edge of Stability
Jeremy M. Cohen, Behrooz Ghorbani, Shankar Krishnan +8
Very little is known about the training dynamics of adaptive gradient methods like Adam in deep learning. In this paper, we shed light on the behavior of these algorithms in the fu…
Pushing the Efficiency-Regret Pareto Frontier for Online Learning of Portfolios and Quantum States
Julian Zimmert, Naman Agarwal, Satyen Kale
We revisit the classical online portfolio selection problem. It is widely assumed that a trade-off between computational complexity and regret is unavoidable, with Cover's Universa…
Machine Learning for Mechanical Ventilation Control (Extended Abstract)
Daniel Suo, Naman Agarwal, Wenhan Xia +12
Mechanical ventilation is one of the most widely used therapies in the ICU. However, despite broad application from anaesthesia to COVID-related life support, many injurious challe…
The Skellam Mechanism for Differentially Private Federated Learning
Naman Agarwal, Peter Kairouz, Ziyu Liu
We introduce the multi-dimensional Skellam mechanism, a discrete differential privacy mechanism based on the difference of two independent Poisson random variables. To quantify its…
Online Target Q-learning with Reverse Experience Replay: Efficiently finding the Optimal Policy for Linear MDPs
Naman Agarwal, Syomantak Chaudhuri, Prateek Jain +2
Q-learning is a popular Reinforcement Learning (RL) algorithm which is widely used in practice with function approximation (Mnih et al., 2015). In contrast, existing theoretical re…
Efficient Methods for Online Multiclass Logistic Regression
Naman Agarwal, Satyen Kale, Julian Zimmert
Multiclass logistic regression is a fundamental task in machine learning with applications in classification and boosting. Previous work (Foster et al., 2018) has highlighted the i…
Acceleration via Fractal Learning Rate Schedules
Naman Agarwal, Surbhi Goel, Cyril Zhang
In practical applications of iterative first-order optimization, the learning rate schedule remains notoriously difficult to understand and expensive to tune. We demonstrate the pr…
A Regret Minimization Approach to Iterative Learning Control
Naman Agarwal, Elad Hazan, Anirudha Majumdar +1
We consider the setting of iterative learning control, or model-based policy learning in the presence of uncertain, time-varying dynamics. In this setting, we propose a new perform…
Deluca -- A Differentiable Control Library: Environments, Methods, and Benchmarking
Paula Gradu, John Hallman, Daniel Suo +7
We present an open-source library of natively differentiable physics and robotics environments, accompanied by gradient-based control methods and a benchmark-ing suite. The introdu…
Machine Learning for Mechanical Ventilation Control
Daniel Suo, Naman Agarwal, Wenhan Xia +12
We consider the problem of controlling an invasive mechanical ventilator for pressure-controlled ventilation: a controller must let air in and out of a sedated patient's lungs acco…
Stochastic Optimization with Laggard Data Pipelines
Naman Agarwal, Rohan Anil, Tomer Koren +2
State-of-the-art optimization is steadily shifting towards massively parallel pipelines with extremely large batch sizes. As a consequence, CPU-bound preprocessing and disk/memory/…
Disentangling Adaptive Gradient Methods from Learning Rates
Naman Agarwal, Rohan Anil, Elad Hazan +2
We investigate several confounding factors in the evaluation of optimization algorithms for deep learning. Primarily, we take a deeper look at how adaptive gradient methods interac…
A Deep Conditioning Treatment of Neural Networks
Naman Agarwal, Pranjal Awasthi, Satyen Kale
We study the role of depth in training randomly initialized overparameterized neural networks. We give a general result showing that depth improves trainability of neural networks…
Logarithmic Regret for Online Control
Naman Agarwal, Elad Hazan, Karan Singh
We study optimal regret bounds for control in linear dynamical systems under adversarially changing strongly convex cost functions, given the knowledge of transition dynamics. This…