336 citations · 628 across the 15 of their papers we have counts for
17 papers
Modulated learning for private and distributed regression with just a single sample per client device
Praneeth Vepakomma, Amirhossein Reisizadeh, Samuel Horváth +1
This work focuses on the question of learning from a large number of devices with each device holding only a single sample of data. Several real-world applications exist to this on…
Collaborative and Efficient Fine-tuning: Leveraging Task Similarity
Gagik Magakyan, Amirhossein Reisizadeh, Chanwoo Park +2
Adaptability has been regarded as a central feature in the foundation models, enabling them to effectively acclimate to unseen downstream tasks. Parameter-efficient fine-tuning met…
A Statistical Physics of Language Model Reasoning
Jack David Carson, Amir Reisizadeh
Transformer LMs show emergent reasoning that resists mechanistic understanding. We offer a statistical physics framework for continuous-time chain-of-thought reasoning dynamics. We…
Robust Decentralized Learning with Local Updates and Gradient Tracking
Sajjad Ghiasvand, Amirhossein Reisizadeh, Mahnoosh Alizadeh +1
As distributed learning applications such as Federated Learning, the Internet of Things (IoT), and Edge Computing grow, it is critical to address the shortcomings of such technolog…
EM for Mixture of Linear Regression with Clustered Data
Amirhossein Reisizadeh, Khashayar Gatmiry, Asuman Ozdaglar
Modern data-driven and distributed learning frameworks deal with diverse massive data generated by clients spread across heterogeneous environments. Indeed, data heterogeneity is a…
Variance-reduced Clipping for Non-convex Optimization
Amirhossein Reisizadeh, Haochuan Li, Subhro Das +1
Gradient clipping is a standard training technique used in deep learning applications such as large-scale language modeling to mitigate exploding gradients. Recent experimental stu…
Gradient Descent for Low-Rank Functions
Romain Cosson, Ali Jadbabaie, Anuran Makur +2
Several recent empirical studies demonstrate that important machine learning tasks, e.g., training deep neural networks, exhibit low-rank structure, where the loss function varies…
An Optimal Transport Approach to Personalized Federated Learning
Farzan Farnia, Amirhossein Reisizadeh, Ramtin Pedarsani +1
Federated learning is a distributed machine learning paradigm, which aims to train a model using the local data of many distributed clients. A key challenge in federated learning i…
Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity
Amirhossein Reisizadeh, Isidoros Tziotis, Hamed Hassani +2
Federated Learning is a novel paradigm that involves learning from data samples distributed across a large network of clients while the data remains local. It is, however, known th…
Robust Federated Learning: The Case of Affine Distribution Shifts
Amirhossein Reisizadeh, Farzan Farnia, Ramtin Pedarsani +1
Federated learning is a distributed paradigm that aims at training models using samples distributed across multiple users in a network while keeping the samples on users' devices w…
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
Amirhossein Reisizadeh, Aryan Mokhtari, Hamed Hassani +2
Federated learning is a distributed framework according to which a model is trained over a set of devices, while keeping data localized. This framework faces several systems-orient…
Robust and Communication-Efficient Collaborative Learning
Amirhossein Reisizadeh, Hossein Taheri, Aryan Mokhtari +2
We consider a decentralized learning problem, where a set of computing nodes aim at solving a non-convex optimization problem collaboratively. It is well-known that decentralized o…
CodedReduce: A Fast and Robust Framework for Gradient Aggregation in Distributed Learning
Amirhossein Reisizadeh, Saurav Prakash, Ramtin Pedarsani +1
We focus on the commonly used synchronous Gradient Descent paradigm for large-scale distributed learning, for which there has been a growing interest to develop efficient and robus…
An Exact Quantized Decentralized Gradient Descent Algorithm
Amirhossein Reisizadeh, Aryan Mokhtari, Hamed Hassani +1
We consider the problem of decentralized consensus optimization, where the sum of smooth and strongly convex functions are minimized over distributed agents that form a con…
Coded Computing for Distributed Graph Analytics
Saurav Prakash, Amirhossein Reisizadeh, Ramtin Pedarsani +1
Performance of distributed graph processing systems significantly suffers from 'communication bottleneck' as a large number of messages are exchanged among servers at each step of…
Latency Analysis of Coded Computation Schemes over Wireless Networks
Amirhossein Reisizadeh, Ramtin Pedarsani
Large-scale distributed computing systems face two major bottlenecks that limit their scalability: straggler delay caused by the variability of computation times at different worke…
Coded Computation over Heterogeneous Clusters
Amirhossein Reisizadeh, Saurav Prakash, Ramtin Pedarsani +1
In large-scale distributed computing clusters, such as Amazon EC2, there are several types of "system noise" that can result in major degradation of performance: bottlenecks due to…