927 citations · 2.7k across the 88 of their papers we have counts for
14 papers · 1 filter
Geometry-Aware Universal Mirror-Prox
Reza Babanezhad, Simon Lacoste-Julien
Mirror-prox (MP) is a well-known algorithm to solve variational inequality (VI) problems. VI with a monotone operator covers a large group of settings such as convex minimization,…
Affine Invariant Analysis of Frank-Wolfe on Strongly Convex Sets
Thomas Kerdreux, Lewis Liu, Simon Lacoste-Julien +1
It is known that the Frank-Wolfe (FW) algorithm, which is affine-covariant, enjoys accelerated convergence rates when the constraint set is strongly convex. However, these results…
On the Convergence of Continuous Constrained Optimization for Structure Learning
Ignavier Ng, Sébastien Lachapelle, Nan Rosemary Ke +2
Recently, structure learning of directed acyclic graphs (DAGs) has been formulated as a continuous optimization problem by leveraging an algebraic characterization of acyclicity. T…
Machine Learning in Airline Crew Pairing to Construct Initial Clusters for Dynamic Constraint Aggregation
Yassine Yaakoubi, François Soumis, Simon Lacoste-Julien
The crew pairing problem (CPP) is generally modelled as a set partitioning problem where the flights have to be partitioned in pairings. A pairing is a sequence of flight legs sepa…
Flight-connection Prediction for Airline Crew Scheduling to Construct Initial Clusters for OR Optimizer
Yassine Yaakoubi, François Soumis, Simon Lacoste-Julien
We present a case study of using machine learning classification algorithms to initialize a large-scale commercial solver (GENCOL) based on column generation in the context of the…
Implicit Regularization via Neural Feature Alignment
Aristide Baratin, Thomas George, César Laurent +4
We approach the problem of implicit regularization in deep learning from a geometrical viewpoint. We highlight a regularization effect induced by a dynamical alignment of the neura…
Stochastic Hamiltonian Gradient Methods for Smooth Games
Nicolas Loizou, Hugo Berard, Alexia Jolicoeur-Martineau +3
The success of adversarial formulations in machine learning has brought renewed motivation for smooth games. In this work, we focus on the class of stochastic Hamiltonian methods a…
Differentiable Causal Discovery from Interventional Data
Philippe Brouillard, Sébastien Lachapelle, Alexandre Lacoste +2
Learning a causal directed acyclic graph from data is a challenging task that involves solving a combinatorial problem for which the solution is not always identifiable. A new line…
Adversarial Example Games
Avishek Joey Bose, Gauthier Gidel, Hugo Berard +4
The existence of adversarial examples capable of fooling trained neural network classifiers calls for a much better understanding of possible attacks to guide the development of sa…
To Each Optimizer a Norm, To Each Norm its Generalization
Sharan Vaswani, Reza Babanezhad, Jose Gallego-Posada +3
We study the implicit regularization of optimization methods for linear models interpolating the training data in the under-parametrized and over-parametrized regimes. Since it is…
Adaptive Gradient Methods Converge Faster with Over-Parameterization (but you should do a line-search)
Sharan Vaswani, Issam Laradji, Frederik Kunstner +3
Adaptive gradient methods are typically used for training over-parameterized models. To better understand their behaviour, we study a simplistic setting -- smooth, convex losses wi…
An Analysis of the Adaptation Speed of Causal Models
Rémi Le Priol, Reza Babanezhad Harikandeh, Yoshua Bengio +1
Consider a collection of datasets generated by unknown interventions on an unknown structural causal model . Recently, Bengio et al. (2020) conjectured that among all candidate…
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence
Nicolas Loizou, Sharan Vaswani, Issam Laradji +1
We propose a stochastic variant of the classical Polyak step-size (Polyak, 1987) commonly used in the subgradient method. Although computing the Polyak step-size requires knowledge…
Accelerating Smooth Games by Manipulating Spectral Shapes
Waïss Azizian, Damien Scieur, Ioannis Mitliagkas +2
We use matrix iteration theory to characterize acceleration in smooth games. We define the spectral shape of a family of games as the set containing all eigenvalues of the Jacobian…