4 papers
Collaborative and Efficient Fine-tuning: Leveraging Task Similarity
Gagik Magakyan, Amirhossein Reisizadeh, Chanwoo Park +2
Adaptability has been regarded as a central feature in the foundation models, enabling them to effectively acclimate to unseen downstream tasks. Parameter-efficient fine-tuning met…
Modulated learning for private and distributed regression with just a single sample per client device
Praneeth Vepakomma, Amirhossein Reisizadeh, Samuel Horváth +1
This work focuses on the question of learning from a large number of devices with each device holding only a single sample of data. Several real-world applications exist to this on…
A Statistical Physics of Language Model Reasoning
Jack David Carson, Amir Reisizadeh
Transformer LMs show emergent reasoning that resists mechanistic understanding. We offer a statistical physics framework for continuous-time chain-of-thought reasoning dynamics. We…
Robust Decentralized Learning with Local Updates and Gradient Tracking
Sajjad Ghiasvand, Amirhossein Reisizadeh, Mahnoosh Alizadeh +1
As distributed learning applications such as Federated Learning, the Internet of Things (IoT), and Edge Computing grow, it is critical to address the shortcomings of such technolog…