6 papers
When and How Unlabeled Data Provably Improve In-Context Learning
Yingcong Li, Xiangyu Chang, Muti Kara +3
Recent research shows that in-context learning (ICL) can be effective even when demonstrations have missing or incorrect labels. To shed light on this capability, we examine a cano…
A Certified Unlearning Approach without Access to Source Data
Umit Yigit Basaran, Sk Miraj Ahmed, Amit Roy-Chowdhury +1
With the growing adoption of data privacy regulations, the ability to erase private or copyrighted information from trained models has become a crucial requirement. Traditional unl…
Mitigating Participation Imbalance Bias in Asynchronous Federated Learning
Xiangyu Chang, Manyi Yao, Srikanth V. Krishnamurthy +5
In Asynchronous Federated Learning (AFL), the central server immediately updates the global model with each arriving client's contribution. As a result, clients perform their local…
Towards Source-Free Machine Unlearning
Sk Miraj Ahmed, Umit Yigit Basaran, Dripta S. Raychaudhuri +5
As machine learning becomes more pervasive and data privacy regulations evolve, the ability to remove private or copyrighted information from trained models is becoming an increasi…
Provable Benefits of Task-Specific Prompts for In-context Learning
Xiangyu Chang, Yingcong Li, Muti Kara +2
The in-context learning capabilities of modern language models have motivated a deeper mathematical understanding of sequence models. A line of recent work has shown that linear at…
Selective Attention: Enhancing Transformer through Principled Context Control
Xuechen Zhang, Xiangyu Chang, Mingchen Li +3
The attention mechanism within the transformer architecture enables the model to weigh and combine tokens based on their relevance to the query. While self-attention has enjoyed ma…