6 papers
When and How Unlabeled Data Provably Improve In-Context Learning
Yingcong Li, Xiangyu Chang, Muti Kara +3
Recent research shows that in-context learning (ICL) can be effective even when demonstrations have missing or incorrect labels. To shed light on this capability, we examine a cano…
Mitigating Participation Imbalance Bias in Asynchronous Federated Learning
Xiangyu Chang, Manyi Yao, Srikanth V. Krishnamurthy +5
In Asynchronous Federated Learning (AFL), the central server immediately updates the global model with each arriving client's contribution. As a result, clients perform their local…
FLASH: Federated Learning Across Simultaneous Heterogeneities
Xiangyu Chang, Sk Miraj Ahmed, Srikanth V. Krishnamurthy +4
The key premise of federated learning (FL) is to train ML models across a diverse set of data-owners (clients), without exchanging local data. An overarching challenge to this date…
Provable Benefits of Task-Specific Prompts for In-context Learning
Xiangyu Chang, Yingcong Li, Muti Kara +2
The in-context learning capabilities of modern language models have motivated a deeper mathematical understanding of sequence models. A line of recent work has shown that linear at…
Selective Attention: Enhancing Transformer through Principled Context Control
Xuechen Zhang, Xiangyu Chang, Mingchen Li +3
The attention mechanism within the transformer architecture enables the model to weigh and combine tokens based on their relevance to the query. While self-attention has enjoyed ma…
CONTRAST: Continual Multi-source Adaptation to Dynamic Distributions
Sk Miraj Ahmed, Fahim Faisal Niloy, Xiangyu Chang +3
Adapting to dynamic data distributions is a practical yet challenging task. One effective strategy is to use a model ensemble, which leverages the diverse expertise of different mo…