164 citations · 411 across the 16 of their papers we have counts for
7 papers · 1 filter
Self-Supervised Learning with Lie Symmetries for Partial Differential Equations
Grégoire Mialon, Quentin Garrido, Hannah Lawrence +3
Machine learning for differential equations paves the way for computationally efficient alternatives to numerical solvers, with potentially broad impacts in science and engineering…
A Cookbook of Self-Supervised Learning
Randall Balestriero, Mark Ibrahim, Vlad Sobal +16
Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning. Yet, much like cooking, training SSL methods is a delicate art wi…
On Inductive Biases for Machine Learning in Data Constrained Settings
Grégoire Mialon
Learning with limited data is one of the biggest problems of machine learning. Current approaches to this issue consist in learning general representations from huge amounts of dat…
Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representations
Grégoire Mialon, Randall Balestriero, Yann LeCun
Self-Supervised Learning (SSL) methods such as VICReg, Barlow Twins or W-MSE avoid collapse of their joint embedding architectures by constraining or regularizing the covariance ma…
GraphiT: Encoding Graph Structure in Transformers
Grégoire Mialon, Dexiong Chen, Margot Selosse +1
We show that viewing graphs as sets of node features and incorporating structural and positional information into a transformer architecture is able to outperform representations l…
A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention
Grégoire Mialon, Dexiong Chen, Alexandre d'Aspremont +1
We address the problem of learning on sets of features, motivated by the need of performing pooling operations in long biological sequences of varying sizes, with long-range depend…
Screening Data Points in Empirical Risk Minimization via Ellipsoidal Regions and Safe Loss Functions
Grégoire Mialon, Alexandre d'Aspremont, Julien Mairal
We design simple screening tests to automatically discard data samples in empirical risk minimization without losing optimization guarantees. We derive loss functions that produce…