704 citations · 4.8k across the 117 of their papers we have counts for
19 papers · 1 filter
Understanding Memorization from the Perspective of Optimization via Efficient Influence Estimation
Futong Liu, Tao Lin, Martin Jaggi
Over-parameterized deep neural networks are able to achieve excellent training accuracy while maintaining a small generalization error. It has also been found that they are able to…
Interpreting Language Models Through Knowledge Graph Extraction
Vinitra Swamy, Angelika Romanou, Martin Jaggi
Transformer-based language models trained on large text corpora have enjoyed immense popularity in the natural language processing community and are commonly used as a starting poi…
Linear Speedup in Personalized Collaborative Learning
El Mahdi Chayti, Sai Praneeth Karimireddy, Sebastian U. Stich +2
Collaborative training can improve the accuracy of a model for a user by trading off the model's bias (introduced by using data from other users who are potentially different) agai…
Optimal Model Averaging: Towards Personalized Collaborative Learning
Felix Grimberg, Mary-Anne Hartley, Sai P. Karimireddy +1
In federated learning, differences in the data or objectives between the participating nodes motivate approaches to train a personalized machine learning model for each node. One s…
WAFFLE: Weighted Averaging for Personalized Federated Learning
Martin Beaussart, Felix Grimberg, Mary-Anne Hartley +1
In federated learning, model personalization can be a very effective strategy to deal with heterogeneous training data across clients. We introduce WAFFLE (Weighted Averaging For F…
RelaySum for Decentralized Deep Learning on Heterogeneous Data
Thijs Vogels, Lie He, Anastasia Koloskova +4
In decentralized machine learning, workers compute model updates on their local data. Because the workers only communicate with few neighbors without central coordination, these up…
On Second-order Optimization Methods for Federated Learning
Sebastian Bischoff, Stephan Günnemann, Martin Jaggi +1
We consider federated learning (FL), where the training data is distributed across a large number of clients. The standard optimization method in this setting is Federated Averagin…
Semantic Perturbations with Normalizing Flows for Improved Generalization
Oguz Kaan Yuksel, Sebastian U. Stich, Martin Jaggi +1
Data augmentation is a widely adopted technique for avoiding overfitting when training deep neural networks. However, this approach requires domain-specific knowledge and is often…
A Field Guide to Federated Optimization
Jianyu Wang, Zachary Charles, Zheng Xu +50
Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy prote…
IFedAvg: Interpretable Data-Interoperability for Federated Learning
David Roschewitz, Mary-Anne Hartley, Luca Corinzia +1
Recently, the ever-growing demand for privacy-oriented machine learning has motivated researchers to develop federated and decentralized learning techniques, allowing individual cl…
Obtaining Better Static Word Embeddings Using Contextual Embedding Models
Prakhar Gupta, Martin Jaggi
The advent of contextual word embeddings -- representations of words which incorporate semantic and syntactic information from their context -- has led to tremendous improvements o…
Implicit Gradient Alignment in Distributed and Federated Learning
Yatin Dandi, Luis Barba, Martin Jaggi
A major obstacle to achieving global convergence in distributed and federated learning is the misalignment of gradients across clients, or mini-batches due to heterogeneity and sto…
Masked Training of Neural Networks with Partial Gradients
Amirkeivan Mohtashami, Martin Jaggi, Sebastian U. Stich
State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters fo…
Lightweight Cross-Lingual Sentence Representation Learning
Zhuoyuan Mao, Prakhar Gupta, Pei Wang +3
Large-scale models for learning fixed-dimensional cross-lingual sentence representations like LASER (Artetxe and Schwenk, 2019b) lead to significant improvement in performance on d…
Federated Learning for Malware Detection in IoT Devices
Valerian Rey, Pedro Miguel Sánchez Sánchez, Alberto Huertas Celdrán +2
This work investigates the possibilities enabled by federated learning concerning IoT malware detection and studies security issues inherent to this new learning paradigm. In this…
Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates
Sebastian U. Stich, Amirkeivan Mohtashami, Martin Jaggi
It has been experimentally observed that the efficiency of distributed training with stochastic gradient (SGD) depends decisively on the batch size and -- in asynchronous implement…
Consensus Control for Decentralized Deep Learning
Lingjing Kong, Tao Lin, Anastasia Koloskova +2
Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters. Experiments in earlier works reveal…
Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data
Tao Lin, Sai Praneeth Karimireddy, Sebastian U. Stich +1
Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of het…
Exact Optimization of Conformal Predictors via Incremental and Decremental Learning
Giovanni Cherubin, Konstantinos Chatzikokolakis, Martin Jaggi
Conformal Predictors (CP) are wrappers around ML models, providing error guarantees under weak assumptions on the data distribution. They are suitable for a wide range of problems,…