139 citations · 423 across the 23 of their papers we have counts for
4 papers · 1 filter
RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks
Xingran Chen, Rohit Bhagat, Ghadir Ayache +3
Parameter-efficient fine-tuning methods such as LoRA have become a standard approach for adapting large foundation models. Adopting fine-tuning to distributed settings faces severa…
The Entrapment Problem in Random Walk Decentralized Learning
Zonghong Liu, Salim El Rouayheb, Matthew Dwyer
This paper explores decentralized learning in a graph-based setting, where data is distributed across nodes. We investigate a decentralized SGD algorithm that utilizes a random wal…
Adaptive Stochastic Gradient Descent for Fast and Communication-Efficient Distributed Learning
Serge Kas Hanna, Rawad Bitar, Parimal Parag +2
We consider the setting where a master wants to run a distributed stochastic gradient descent (SGD) algorithm on workers, each having a subset of the data. Distributed SGD may…
Adaptive Distributed Stochastic Gradient Descent for Minimizing Delay in the Presence of Stragglers
Serge Kas Hanna, Rawad Bitar, Parimal Parag +2
We consider the setting where a master wants to run a distributed stochastic gradient descent (SGD) algorithm on workers each having a subset of the data. Distributed SGD may s…