40 citations · 97 across the 16 of their papers we have counts for
16 papers
Training Dynamics of Softmax Self-Attention: Fast Global Convergence via Preconditioning
Gautam Goel, Mahdi Soltanolkotabi, Peter Bartlett
We study the training dynamics of gradient descent in a softmax self-attention layer trained to perform linear regression and show that a simple first-order optimization algorithm…
Can a Transformer Represent a Kalman Filter?
Gautam Goel, Peter Bartlett
Transformers are a class of autoregressive deep learning architectures which have recently achieved state-of-the-art performance in various vision, language, and robotics tasks. We…
Best of Both Worlds in Online Control: Competitive Ratio and Policy Regret
Gautam Goel, Naman Agarwal, Karan Singh +1
We consider the fundamental problem of online control of a linear dynamical system from two different viewpoints: regret minimization and competitive analysis. We prove that the op…
Measurement-Feedback Control with Optimal Data-Dependent Regret
Gautam Goel, Babak Hassibi
Inspired by online learning, data-dependent regret has recently been proposed as a criterion for controller design. In the regret-optimal control paradigm, causal controllers are d…
Online estimation and control with optimal pathlength regret
Gautam Goel, Babak Hassibi
A natural goal when designing online learning algorithms for non-stationary environments is to bound the regret of the algorithm in terms of the temporal variation of the input seq…
Competitive Control
Gautam Goel, Babak Hassibi
We consider control from the perspective of competitive analysis. Unlike much prior work on learning-based control, which focuses on minimizing regret against the best controller s…
Regret-optimal Estimation and Control
Gautam Goel, Babak Hassibi
We consider estimation and control in linear time-varying dynamical systems from the perspective of regret minimization. Unlike most prior work in this area, we focus on the proble…
Regret-Optimal LQR Control
Oron Sabag, Gautam Goel, Sahin Lale +1
We consider the infinite-horizon LQR control problem. Motivated by competitive analysis in online learning, as a criterion for controller design we introduce the dynamic regret, de…
Regret-optimal measurement-feedback control
Gautam Goel, Babak Hassibi
We consider measurement-feedback control in linear dynamical systems from the perspective of regret minimization. Unlike most prior work in this area, we focus on the problem of de…
Regret-optimal control in dynamic environments
Gautam Goel, Babak Hassibi
We consider control in linear time-varying dynamical systems from the perspective of regret minimization. Unlike most prior work in this area, we focus on the problem of designing…
The Power of Linear Controllers in LQR Control
Gautam Goel, Babak Hassibi
The Linear Quadratic Regulator (LQR) framework considers the problem of regulating a linear dynamical system perturbed by environmental noise. We compute the policy regret between…
Online Optimization with Predictions and Non-convex Losses
Yiheng Lin, Gautam Goel, Adam Wierman
We study online optimization in a setting where an online learner seeks to optimize a per-round hitting cost, which may be non-convex, while incurring a movement cost when changing…
Beyond Online Balanced Descent: An Optimal Algorithm for Smoothed Online Optimization
Gautam Goel, Yiheng Lin, Haoyuan Sun +1
We study online convex optimization in a setting where the learner seeks to minimize the sum of a per-round hitting cost and a movement cost which is incurred when changing decisio…
Smoothed Online Optimization for Regression and Control
Gautam Goel, Adam Wierman
We consider Online Convex Optimization (OCO) in the setting where the costs are -strongly convex and the online learner pays a switching cost for changing decisions between roun…
Smoothed Online Convex Optimization in High Dimensions via Online Balanced Descent
Niangjun Chen, Gautam Goel, Adam Wierman
We study Smoothed Online Convex Optimization, a version of online convex optimization where the learner incurs a penalty for changing her actions between rounds. Given a $Ω(\sqrt{d…
Thinking Fast and Slow: Optimization Decomposition Across Timescales
Gautam Goel, Niangjun Chen, Adam Wierman
Many real-world control systems, such as the smart grid and human sensorimotor control systems, have decentralized components that react quickly using local information and central…