21 citations · 136 across the 22 of their papers we have counts for
23 papers
Convergence of projected stochastic natural gradient variational inference for various step size and sample or batch size schedules
Thomas Guilmeau, Hadrien Hendrikx, Florence Forbes
Stochastic natural gradient variational inference (NGVI) is a popular and efficient algorithm for Bayesian inference. Despite empirical success, the convergence of this method is s…
From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity
Renaud Gaucher, Aymeric Dieuleveut, Hadrien Hendrikx
Standard federated learning algorithms are vulnerable to adversarial nodes, a.k.a. Byzantine failures. To solve this issue, robust distributed learning algorithms have been develop…
BotaCLIP: Contrastive Learning for Botany-Aware Representation of Earth Observation Data
Selene Cerna, Sara Si-Moussi, Wilfried Thuiller +2
Foundation models have demonstrated a remarkable ability to learn rich, transferable representations across diverse modalities such as images, text, and audio. In modern machine le…
Exponential Moving Average of Weights in Deep Learning: Dynamics and Benefits
Daniel Morales-Brotons, Thijs Vogels, Hadrien Hendrikx
Weight averaging of Stochastic Gradient Descent (SGD) iterates is a popular method for training deep learning models. While it is often used as part of complex training pipelines t…
Unified Breakdown Analysis for Byzantine Robust Gossip
Renaud Gaucher, Aymeric Dieuleveut, Hadrien Hendrikx
In decentralized machine learning, different devices communicate in a peer-to-peer manner to collaboratively learn from each other's data. Such approaches are vulnerable to misbeha…
Byzantine-Robust Gossip: Insights from a Dual Approach
Renaud Gaucher, Aymeric Dieuleveut, Hadrien Hendrikx
Distributed learning has many computational benefits but is vulnerable to attacks from a subset of devices transmitting incorrect information. This paper investigates Byzantine-res…
Investigating Variance Definitions for Mirror Descent with Relative Smoothness
Hadrien Hendrikx
Mirror Descent is a popular algorithm, that extends Gradients Descent (GD) beyond the Euclidean geometry. One of its benefits is to enable strong convergence guarantees through smo…
The Relative Gaussian Mechanism and its Application to Private Gradient Descent
Hadrien Hendrikx, Paul Mangold, Aurélien Bellet
The Gaussian Mechanism (GM), which consists in adding Gaussian noise to a vector-valued query before releasing it, is a standard privacy protection mechanism. In particular, given…
Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees
Anastasia Koloskova, Hadrien Hendrikx, Sebastian U. Stich
Gradient clipping is a popular modification to standard (stochastic) gradient descent, at every iteration limiting the gradient norm to a certain value . It is widely used fo…
Beyond spectral gap (extended): The role of the topology in decentralized learning
Thijs Vogels, Hadrien Hendrikx, Martin Jaggi
In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rat…
Beyond spectral gap: The role of the topology in decentralized learning
Thijs Vogels, Hadrien Hendrikx, Martin Jaggi
In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rat…
A principled framework for the design and analysis of token algorithms
Hadrien Hendrikx
We consider a decentralized optimization problem, in which nodes collaborate to optimize a global objective function using local communications only. While many decentralized a…
Asynchronous speedup in decentralized optimization
Mathieu Even, Hadrien Hendrikx, Laurent Massoulie
In decentralized optimization, nodes of a communication network each possess a local objective function, and communicate using gossip-based methods in order to minimize the average…
Fast Stochastic Bregman Gradient Methods: Sharp Analysis and Variance Reduction
Radu-Alexandru Dragomir, Mathieu Even, Hadrien Hendrikx
We study the problem of minimizing a relatively-smooth convex function using stochastic Bregman gradient methods. We first prove the convergence of Bregman Stochastic Gradient Desc…
Asynchrony and Acceleration in Gossip Algorithms
Mathieu Even, Hadrien Hendrikx, Laurent Massoulié
This paper considers the minimization of a sum of smooth and strongly convex functions dispatched over the nodes of a communication network. Previous works on the subject either fo…
Dual-Free Stochastic Decentralized Optimization with Variance Reduction
Hadrien Hendrikx, Francis Bach, Laurent Massoulié
We consider the problem of training machine learning models on distributed data in a decentralized way. For finite-sum problems, fast single-machine algorithms for large datasets r…
An Optimal Algorithm for Decentralized Finite Sum Optimization
Hadrien Hendrikx, Francis Bach, Laurent Massoulie
Modern large-scale finite-sum optimization relies on two key aspects: distribution and stochastic updates. For smooth and strongly convex problems, existing decentralized algorithm…
Statistically Preconditioned Accelerated Gradient Method for Distributed Optimization
Hadrien Hendrikx, Lin Xiao, Sebastien Bubeck +2
We consider the setting of distributed empirical risk minimization where multiple machines compute the gradients in parallel and a centralized server updates the model parameters.…
An Accelerated Decentralized Stochastic Proximal Algorithm for Finite Sums
Hadrien Hendrikx, Francis Bach, Laurent Massoulie
Modern large-scale finite-sum optimization relies on two key aspects: distribution and stochastic updates. For smooth and strongly convex problems, existing decentralized algorithm…
Who started this rumor? Quantifying the natural differential privacy guarantees of gossip protocols
Aurélien Bellet, Rachid Guerraoui, Hadrien Hendrikx
Gossip protocols are widely used to disseminate information in massive peer-to-peer networks. These protocols are often claimed to guarantee privacy because of the uncertainty they…
Asynchronous Accelerated Proximal Stochastic Gradient for Strongly Convex Distributed Finite Sums
Hadrien Hendrikx, Francis Bach, Laurent Massoulié
In this work, we study the problem of minimizing the sum of strongly convex functions split over a network of nodes. We propose the decentralized and asynchronous algorithm ADF…
Accelerated Decentralized Optimization with Local Updates for Smooth and Strongly Convex Objectives
Hadrien Hendrikx, Francis Bach, Laurent Massoulié
In this paper, we study the problem of minimizing a sum of smooth and strongly convex functions split over the nodes of a network in a decentralized fashion. We propose the algorit…
Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning
El Mahdi El Mhamdi, Rachid Guerraoui, Hadrien Hendrikx +1
In reinforcement learning, agents learn by performing actions and observing their outcomes. Sometimes, it is desirable for a human operator to \textit{interrupt} an agent in order…