1 citations · 1 across the 4 of their papers we have counts for
4 papers · 1 filter
SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning
Yongkang Liu, Xing Li, Mengjie Zhao +7
As the number of model parameters increases, parameter-efficient fine-tuning (PEFT) has become the go-to choice for tailoring pre-trained large language models. Low-rank Adaptation…
Neural Operators for Multi-Task Control and Adaptation
David Sewell, Xingjian Li, Stepan Tretiakov +2
Neural operator methods have emerged as powerful tools for learning mappings between infinite-dimensional function spaces, yet their potential in optimal control remains largely un…
Optimal Control for Transformer Architectures: Enhancing Generalization, Robustness and Efficiency
Kelvin Kan, Xingjian Li, Benjamin J. Zhang +3
We study Transformers through the perspective of optimal control theory, using tools from continuous-time formulations to derive actionable insights into training and architecture…
OT-Transformer: A Continuous-time Transformer Architecture with Optimal Transport Regularization
Kelvin Kan, Xingjian Li, Stanley Osher
Transformers have achieved state-of-the-art performance in numerous tasks. In this paper, we propose a continuous-time formulation of transformers. Specifically, we consider a dyna…