732 citations · 1.7k across the 44 of their papers we have counts for
31 papers · 1 filter
A Closer Look at Personalized Fine-Tuning in Heterogeneous Federated Learning
Minghui Chen, Hrad Ghoukasian, Ruinan Jin +3
Federated Learning (FL) enables decentralized, privacy-preserving model training but struggles to balance global generalization and local personalization due to non-identical data…
f-INE: A Hypothesis Testing Framework for Estimating Influence under Training Randomness
Subhodip Panda, Dhruv Tarsadiya, Shashwat Sourav +2
Influence estimation methods promise to explain and debug machine learning by estimating the impact of individual samples on the final model. Yet, existing methods collapse under t…
Conformal Prediction Adaptive to Unknown Subpopulation Shifts
Nien-Shao Wang, Duygu Nur Yaldiz, Yavuz Faruk Bakman +1
Conformal prediction is widely used to equip black-box machine learning models with uncertainty quantification, offering formal coverage guarantees under exchangeable data. However…
Reconsidering LLM Uncertainty Estimation Methods in the Wild
Yavuz Bakman, Duygu Nur Yaldiz, Sungmin Kang +4
Large Language Model (LLM) Uncertainty Estimation (UE) methods have become a crucial tool for detecting hallucinations in recent years. While numerous UE methods have been proposed…
DAVED: Data Acquisition via Experimental Design for Data Markets
Charles Lu, Baihe Huang, Sai Praneeth Karimireddy +3
The acquisition of training data is crucial for machine learning applications. Data markets can increase the supply of data, particularly in data-scarce domains such as healthcare,…
Scaff-PD: Communication Efficient Fair and Robust Federated Learning
Yaodong Yu, Sai Praneeth Karimireddy, Yi Ma +1
We present Scaff-PD, a fast and communication-efficient algorithm for distributionally robust federated learning. Our approach improves fairness by optimizing a family of distribut…
Provably Personalized and Robust Federated Learning
Mariel Werner, Lie He, Michael Jordan +2
Identifying clients with similar objectives and learning a model-per-cluster is an intuitive and interpretable approach to personalization in federated learning. However, doing so…
Federated Conformal Predictors for Distributed Uncertainty Quantification
Charles Lu, Yaodong Yu, Sai Praneeth Karimireddy +2
Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning since it can be easily applied as a post-processing ste…
FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings
Jean Ogier du Terrail, Samy-Safwan Ayed, Edwige Cyffers +21
Federated Learning (FL) is a novel approach enabling several clients holding sensitive data to collaboratively train machine learning models, without centralizing data. The cross-s…
TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernels
Yaodong Yu, Alexander Wei, Sai Praneeth Karimireddy +2
State-of-the-art federated learning methods can perform far worse than their centralized counterparts when clients have dissimilar data distributions. For neural networks, even whe…
Optimization with Access to Auxiliary Information
El Mahdi Chayti, Sai Praneeth Karimireddy
We investigate the fundamental optimization question of minimizing a target function , whose gradients are expensive to compute or have limited availability, given access to som…
Agree to Disagree: Diversity through Disagreement for Better Transferability
Matteo Pagliardini, Martin Jaggi, François Fleuret +1
Gradient-based learning algorithms have an implicit simplicity bias which in effect can limit the diversity of predictors being sampled by the learning procedure. This behavior can…
Byzantine-Robust Decentralized Learning via ClippedGossip
Lie He, Sai Praneeth Karimireddy, Martin Jaggi
In this paper, we study the challenging task of Byzantine-robust decentralized training on arbitrary communication graphs. Unlike federated learning where workers communicate throu…
Linear Speedup in Personalized Collaborative Learning
El Mahdi Chayti, Sai Praneeth Karimireddy, Sebastian U. Stich +2
Collaborative training can improve the accuracy of a model for a user by trading off the model's bias (introduced by using data from other users who are potentially different) agai…
Optimal Model Averaging: Towards Personalized Collaborative Learning
Felix Grimberg, Mary-Anne Hartley, Sai P. Karimireddy +1
In federated learning, differences in the data or objectives between the participating nodes motivate approaches to train a personalized machine learning model for each node. One s…
Towards Model Agnostic Federated Learning Using Knowledge Distillation
Andrei Afonin, Sai Praneeth Karimireddy
Is it possible to design an universal API for federated learning using which an ad-hoc group of data-holders (agents) collaborate with each other and perform federated learning? Su…
RelaySum for Decentralized Deep Learning on Heterogeneous Data
Thijs Vogels, Lie He, Anastasia Koloskova +4
In decentralized machine learning, workers compute model updates on their local data. Because the workers only communicate with few neighbors without central coordination, these up…
A Field Guide to Federated Optimization
Jianyu Wang, Zachary Charles, Zheng Xu +50
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy prote…
Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data
Tao Lin, Sai Praneeth Karimireddy, Sebastian U. Stich +1
Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of het…
Learning from History for Byzantine Robust Optimization
Sai Praneeth Karimireddy, Lie He, Martin Jaggi
Byzantine robustness has received significant attention recently given its importance for distributed and federated learning. In spite of this, we identify severe flaws in existing…
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning
Sai Praneeth Karimireddy, Martin Jaggi, Satyen Kale +4
Federated learning (FL) is a challenging setting for optimization due to the heterogeneity of the data across different clients which gives rise to the client drift phenomenon. In…
PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning
Thijs Vogels, Sai Praneeth Karimireddy, Martin Jaggi
Lossy gradient compression has become a practical tool to overcome the communication bottleneck in centrally coordinated distributed training of machine learning models. However, a…
Byzantine-Robust Learning on Heterogeneous Datasets via Bucketing
Sai Praneeth Karimireddy, Lie He, Martin Jaggi
In Byzantine robust distributed or federated learning, a central server wants to train a machine learning model over data distributed across multiple workers. However, a fraction o…
Secure Byzantine-Robust Machine Learning
Lie He, Sai Praneeth Karimireddy, Martin Jaggi
Increasingly machine learning systems are being deployed to edge servers and devices (e.g. mobile phones) and trained in a collaborative manner. Such distributed/federated/decentra…