63.6k citations · 104.5k across the 42 of their papers we have counts for
20 papers · 1 filter
Stability-Aware Training of Machine Learning Force Fields with Differentiable Boltzmann Estimators
Sanjeev Raja, Ishan Amin, Fabian Pedregosa +1
Machine learning force fields (MLFFs) are an attractive alternative to ab-initio methods for molecular dynamics (MD) simulations. However, they can produce unstable simulations, li…
On the Interplay Between Stepsize Tuning and Progressive Sharpening
Vincent Roulet, Atish Agarwala, Fabian Pedregosa
Recent empirical work has revealed an intriguing property of deep learning models by which the sharpness (largest eigenvalue of the Hessian) increases throughout optimization until…
On Implicit Bias in Overparameterized Bilevel Optimization
Paul Vicol, Jonathan Lorraine, Fabian Pedregosa +2
Many problems in machine learning involve bilevel optimization (BLO), including hyperparameter optimization, meta-learning, and dataset distillation. Bilevel problems consist of tw…
A Novel Stochastic Gradient Descent Algorithm for Learning Principal Subspaces
Charline Le Lan, Joshua Greaves, Jesse Farebrother +4
Many machine learning problems encode their data as a matrix with a possibly very large number of rows and columns. In several applications like neuroscience, image compression or…
When is Momentum Extragradient Optimal? A Polynomial-Based Analysis
Junhyung Lyle Kim, Gauthier Gidel, Anastasios Kyrillidis +1
The extragradient method has gained popularity due to its robust convergence properties for differentiable games. Unlike single-objective optimization, game dynamics involve comple…
Second-order regression models exhibit progressive sharpening to the edge of stability
Atish Agarwala, Fabian Pedregosa, Jeffrey Pennington
Recent studies of gradient descent with large step sizes have shown that there is often a regime with an initial increase in the largest eigenvalue of the loss Hessian (progressive…
Cutting Some Slack for SGD with Adaptive Polyak Stepsizes
Robert M. Gower, Mathieu Blondel, Nidham Gazagnadou +1
Tuning the step size of stochastic gradient descent is tedious and error prone. This has motivated the development of methods that automatically adapt the step size using readily a…
GradMax: Growing Neural Networks using Gradient Information
Utku Evci, Bart van Merriënboer, Thomas Unterthiner +2
The architecture and the parameters of neural networks are often optimized independently, which requires costly retraining of the parameters whenever the architecture is modified.…
Boosting Variational Inference With Locally Adaptive Step-Sizes
Gideon Dresdner, Saurav Shekhar, Fabian Pedregosa +2
Variational Inference makes a trade-off between the capacity of the variational family and the tractability of finding an approximate posterior distribution. Instead, Boosting Vari…
Efficient and Modular Implicit Differentiation
Mathieu Blondel, Quentin Berthet, Marco Cuturi +5
Automatic differentiation (autodiff) has revolutionized machine learning. It allows to express complex computations by composing elementary ones in creative ways and removes the bu…
Bridging the Gap Between Adversarial Robustness and Optimization Bias
Fartash Faghri, Sven Gowal, Cristina Vasconcelos +3
We demonstrate that the choice of optimizer, neural network architecture, and regularizer significantly affect the adversarial robustness of linear neural networks, providing guara…
The Geometry of Sign Gradient Descent
Lukas Balles, Fabian Pedregosa, Nicolas Le Roux
Sign-based optimization methods have become popular in machine learning due to their favorable communication cost in distributed optimization and their surprisingly good performanc…
The Difficulty of Training Sparse Neural Networks
Utku Evci, Fabian Pedregosa, Aidan Gomez +1
We investigate the difficulties of training sparse neural networks and make new observations about optimization dynamics and the energy landscape within the sparse regime. Recent w…
On the interplay between noise and curvature and its effect on optimization and generalization
Valentin Thomas, Fabian Pedregosa, Bart van Merriënboer +3
The speed at which one can minimize an expected loss using stochastic methods depends on two properties: the curvature of the loss and the variance of the gradients. While most pre…
Machine Learning for Neuroimaging with Scikit-Learn
Alexandre Abraham, Fabian Pedregosa, Michael Eickenberg +6
Statistical machine learning methods are increasingly used for neuroimaging data analysis. Their main virtue is their ability to model high-dimensional datasets, e.g. multivariate…
On the Consistency of Ordinal Regression Methods
Fabian Pedregosa, Francis Bach, Alexandre Gramfort
Many of the ordinal regression models that have been proposed in the literature can be seen as methods that minimize a convex surrogate of the zero-one, absolute, or squared loss f…
HRF estimation improves sensitivity of fMRI encoding and decoding models
Fabian Pedregosa, Michael Eickenberg, Bertrand Thirion +1
Extracting activation patterns from functional Magnetic Resonance Images (fMRI) datasets remains challenging in rapid-event designs due to the inherent delay of blood oxygen level-…
Improved brain pattern recovery through ranking approaches
Fabian Pedregosa, Alexandre Gramfort, Gaël Varoquaux +3
Inferring the functional specificity of brain regions from functional Magnetic Resonance Images (fMRI) data is a challenging statistical problem. While the General Linear Model (GL…
Learning to rank from medical imaging data
Fabian Pedregosa, Alexandre Gramfort, Gaël Varoquaux +3
Medical images can be used to predict a clinical score coding for the severity of a disease, a pain level or the complexity of a cognitive task. In all these cases, the predicted v…
Scikit-learn: Machine Learning in Python
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort +16
Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses…