4 papers
On the Role of Transformer Feed-Forward Layers in Nonlinear In-Context Learning
Haoyuan Sun, Ali Jadbabaie, Navid Azizan
Transformer-based models demonstrate a remarkable ability for in-context learning (ICL), where they can adapt to unseen tasks from a few prompt examples without parameter updates.…
Meta-Learning for Adaptive Control with Automated Mirror Descent
Sunbochen Tang, Haoyuan Sun, Navid Azizan
Adaptive control achieves concurrent parameter learning and stable control under uncertainties that are linearly parameterized with known nonlinear features. Nonetheless, it is oft…
A least-square method for non-asymptotic identification in linear switching control
Haoyuan Sun, Ali Jadbabaie
The focus of this paper is on linear system identification in the setting where it is known that the underlying partially-observed linear dynamical system lies within a finite coll…
Private Synthetic Data Meets Ensemble Learning
Haoyuan Sun, Navid Azizan, Akash Srivastava +1
When machine learning models are trained on synthetic data and then deployed on real data, there is often a performance drop due to the distribution shift between synthetic and rea…