538 citations · 1.1k across the 31 of their papers we have counts for
33 papers
Unraveling the Interconnected Axes of Heterogeneity in Machine Learning for Democratic and Inclusive Advancements
Maryam Molamohammadi, Afaf Taik, Nicolas Le Roux +1
The growing utilization of machine learning (ML) in decision-making processes raises questions about its benefits to society. In this study, we identify and analyze three axes of h…
Joint Prompt Optimization of Stacked LLMs using Variational Inference
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté +6
Large language models (LLMs) can be seen as atomic units of computation mapping sequences to a distribution over sequences. Thus, they can be seen as stochastic language layers in…
Decision-Aware Actor-Critic with Function Approximation and Theoretical Guarantees
Sharan Vaswani, Amirreza Kazemi, Reza Babanezhad +1
Actor-critic (AC) methods are widely used in reinforcement learning (RL) and benefit from the flexibility of using any policy gradient method as the actor and value-based method as…
Target-based Surrogates for Stochastic Optimization
Jonathan Wilder Lavington, Sharan Vaswani, Reza Babanezhad +2
We consider minimizing functions for which it is expensive to compute the (possibly stochastic) gradient. Such functions are prevalent in reinforcement learning, imitation learning…
Multi-Head Adapter Routing for Cross-Task Generalization
Lucas Caccia, Edoardo Ponti, Zhan Su +3
Parameter-efficient fine-tuning (PEFT) for cross-task generalization consists in pre-training adapters on a multi-task training set before few-shot adaptation to test tasks. Polytr…
Impact of Aliasing on Generalization in Deep Convolutional Networks
Cristina Vasconcelos, Hugo Larochelle, Vincent Dumoulin +3
We investigate the impact of aliasing on generalization in Deep Convolutional Networks and show that data augmentation schemes alone are unable to prevent it due to structural limi…
A general class of surrogate functions for stable and efficient reinforcement learning
Sharan Vaswani, Olivier Bachem, Simone Totaro +6
Common policy gradient methods rely on the maximization of a sequence of surrogate functions. In recent years, many such surrogate functions have been proposed, most without strong…
On the Convergence of Stochastic Extragradient for Bilinear Games using Restarted Iteration Averaging
Chris Junchi Li, Yaodong Yu, Nicolas Loizou +4
We study the stochastic bilinear minimax optimization problem, presenting an analysis of the same-sample Stochastic ExtraGradient (SEG) method with constant step size, and presenti…
Bridging the Gap Between Adversarial Robustness and Optimization Bias
Fartash Faghri, Sven Gowal, Cristina Vasconcelos +3
We demonstrate that the choice of optimizer, neural network architecture, and regularizer significantly affect the adversarial robustness of linear neural networks, providing guara…
An Effective Anti-Aliasing Approach for Residual Networks
Cristina Vasconcelos, Hugo Larochelle, Vincent Dumoulin +2
Image pre-processing in the frequency domain has traditionally played a vital role in computer vision and was even part of the standard pipeline in the early days of deep learning.…
Beyond variance reduction: Understanding the true impact of baselines on policy optimization
Wesley Chung, Valentin Thomas, Marlos C. Machado +1
Bandit and reinforcement learning (RL) problems can often be framed as optimization problems where the goal is to maximize average performance while having access only to stochasti…
To Each Optimizer a Norm, To Each Norm its Generalization
Sharan Vaswani, Reza Babanezhad, Jose Gallego-Posada +3
We study the implicit regularization of optimization methods for linear models interpolating the training data in the under-parametrized and over-parametrized regimes. Since it is…
An operator view of policy gradient methods
Dibya Ghosh, Marlos C. Machado, Nicolas Le Roux
We cast policy gradient methods as the repeated application of two operators: a policy improvement operator , which maps any policy to a better one ,…
The Geometry of Sign Gradient Descent
Lukas Balles, Fabian Pedregosa, Nicolas Le Roux
Sign-based optimization methods have become popular in machine learning due to their favorable communication cost in distributed optimization and their surprisingly good performanc…
Reducing the variance in online optimization by transporting past gradients
Sébastien M. R. Arnold, Pierre-Antoine Manzagol, Reza Babanezhad +2
Most stochastic optimization methods use gradients once before discarding them. While variance reduction methods have shown that reusing past gradients can be beneficial when there…
On the interplay between noise and curvature and its effect on optimization and generalization
Valentin Thomas, Fabian Pedregosa, Bart van Merriënboer +3
The speed at which one can minimize an expected loss using stochastic methods depends on two properties: the curvature of the loss and the variance of the gradients. While most pre…
Anytime Tail Averaging
Nicolas Le Roux
Tail averaging consists in averaging the last examples in a stream. Common techniques either have a memory requirement which grows with the number of samples to average, are not av…
Distributional reinforcement learning with linear function approximation
Marc G. Bellemare, Nicolas Le Roux, Pablo Samuel Castro +1
Despite many algorithmic advances, our theoretical understanding of practical distributional reinforcement learning methods remains limited. One exception is Rowland et al. (2018)'…
Negative eigenvalues of the Hessian in deep neural networks
Guillaume Alain, Nicolas Le Roux, Pierre-Antoine Manzagol
The loss function of deep networks is known to be non-convex but the precise nature of this nonconvexity is still an active area of research. In this work, we study the loss landsc…
A Geometric Perspective on Optimal Representations for Reinforcement Learning
Marc G. Bellemare, Will Dabney, Robert Dadashi +6
We propose a new perspective on representation learning in reinforcement learning based on geometric properties of the space of value functions. We leverage this perspective to pro…
The Value Function Polytope in Reinforcement Learning
Robert Dadashi, Adrien Ali Taïga, Nicolas Le Roux +2
We establish geometric and topological properties of the space of value functions in finite state-action Markov decision processes. Our main contribution is the characterization of…
Understanding the impact of entropy on policy optimization
Zafarali Ahmed, Nicolas Le Roux, Mohammad Norouzi +1
Entropy regularization is commonly used to improve policy optimization in reinforcement learning. It is believed to help with \emph{exploration} by encouraging the selection of mor…
Tracking the gradients using the Hessian: A new look at variance reducing stochastic methods
Robert M. Gower, Nicolas Le Roux, Francis Bach
Our goal is to improve variance reducing stochastic methods through better control variates. We first propose a modification of SVRG which uses the Hessian to track gradients over…
Distributed SAGA: Maintaining linear convergence rate with limited communication
Clément Calauzènes, Nicolas Le Roux
In recent years, variance-reducing stochastic methods have shown great practical performance, exhibiting linear convergence rate when other stochastic methods offered a sub-linear…