16 papers
Exact Flow Linear Attention: Exact Solution from Continuous-Time Dynamics
Jingdi Lei, Di Zhang, Soujanya Poria
In this paper, we introduce Exact Flow Linear Attention~(EFLA), an exact-flow formulation of delta-rule linear attention. We show that the delta-rule update can be interpreted as a…
Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adapter
Weizhi Zhong, Huan Yang, Zheng Liu +5
Personalized text-to-image generation aims to synthesize images of user-provided concepts in diverse contexts. Despite recent progress in multi-concept personalization, most are li…
Bayesian Optimality of In-Context Learning with Selective State Spaces
Di Zhang, Jiaqi Xing
We propose Bayesian optimal sequential prediction as a new principle for understanding in-context learning (ICL). Unlike interpretations framing Transformers as performing implicit…
Bigger Is Safer: Provable Robustness in In-Context Learning Scales with Capacity
Di Zhang, Ningxu Zhang, Zimeng Liu
In-context learning (ICL) allows large language models to adapt to new tasks from a few examples without updating their parameters. Existing theories explain ICL by assuming the te…
AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
Can Jin, Yang Zhou, Qixin Zhang +8
Test-time scaling strategies for Large Language Models predominantly rely on either reinforcement learning with sparse outcome rewards or search-based methods guided by static Proc…
Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts
Di Zhang, Xun Wu, Shaohan Huang +6
Recent advances in reinforcement learning (RL) have substantially improved the training of large-scale language models, leading to significant gains in generation quality and reaso…