154 citations · 861 across the 94 of their papers we have counts for
12 papers · 1 filter
Mitigating Participation Imbalance Bias in Asynchronous Federated Learning
Xiangyu Chang, Manyi Yao, Srikanth V. Krishnamurthy +5
In Asynchronous Federated Learning (AFL), the central server immediately updates the global model with each arriving client's contribution. As a result, clients perform their local…
Attention with Trained Embeddings Provably Selects Important Tokens
Diyuan Wu, Aleksandr Shevchenko, Samet Oymak +1
Token embeddings play a crucial role in language modeling but, despite this practical relevance, their theoretical understanding remains limited. Our paper addresses the gap by cha…
BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning
Xuechen Zhang, Zijian Huang, Yingcong Li +3
Small language models (SLMs) struggle to learn complex reasoning behaviors, especially when high-quality traces are scarce or difficult to learn from. The standard training approac…
When and How Unlabeled Data Provably Improve In-Context Learning
Yingcong Li, Xiangyu Chang, Muti Kara +3
Recent research shows that in-context learning (ICL) can be effective even when demonstrations have missing or incorrect labels. To shed light on this capability, we examine a cano…
Extrapolation by Association: Length Generalization Transfer in Transformers
Ziyang Cai, Nayoung Lee, Avi Schwarzschild +2
Transformer language models have demonstrated impressive generalization capabilities in natural language domains, yet we lack a fine-grained understanding of how such generalizatio…
Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcement
Xuechen Zhang, Zijian Huang, Chenshun Ni +3
Recent research enhances language model reasoning by scaling test-time compute via longer chain-of-thought traces. This often improves accuracy but also introduces redundancy and h…