44 citations · 89 across the 19 of their papers we have counts for
19 papers
Generating Tabular Data Using Heterogeneous Sequential Feature Forest Flow Matching
Ange-Clément Akazan, Alexia Jolicoeur-Martineau, Ioannis Mitliagkas
Privacy and regulatory constraints make data generation vital to advancing machine learning without relying on real-world datasets. A leading approach for tabular data generation i…
Understanding Adam Requires Better Rotation Dependent Assumptions
Tianyue H. Zhang, Lucas Maes, Alan Milligan +5
Despite its widespread adoption, Adam's advantage over Stochastic Gradient Descent (SGD) lacks a comprehensive theoretical explanation. This paper investigates Adam's sensitivity t…
Compositional Risk Minimization
Divyat Mahajan, Mohammad Pezeshki, Charles Arnal +3
Compositional generalization is a crucial step towards developing data-efficient intelligent machines that generalize in human-like ways. In this work, we tackle a challenging form…
Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection
Charles Guille-Escuret, Pierre-André Noël, Ioannis Mitliagkas +2
Improving the reliability of deployed machine learning systems often involves developing methods to detect out-of-distribution (OOD) inputs. However, existing research often narrow…
An Empirical Study of Pre-trained Model Selection for Out-of-Distribution Generalization and Calibration
Hiroki Naganuma, Ryuichiro Hataya, Kotaro Yoshida +1
In the field of computer vision, fine-tuning pre-trained models has become a prevalent strategy for out-of-distribution (OOD) generalization tasks. Different from most prior work t…
Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation
Sébastien Lachapelle, Divyat Mahajan, Ioannis Mitliagkas +1
We tackle the problems of latent variables identification and ``out-of-support'' image generation in representation learning. We show that both are possible for a class of decoders…
No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths
Charles Guille-Escuret, Hiroki Naganuma, Kilian Fatras +1
Understanding the optimization dynamics of neural networks is necessary for closing the gap between theory and practice. Stochastic first-order optimization algorithms are known to…
Performative Prediction with Neural Networks
Mehrnaz Mofakhami, Ioannis Mitliagkas, Gauthier Gidel
Performative prediction is a framework for learning models that influence the data they intend to predict. We focus on finding classifiers that are performatively stable, i.e. opti…
Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning
Sébastien Lachapelle, Tristan Deleu, Divyat Mahajan +4
Although disentangled representations are often said to be beneficial for downstream tasks, current empirical and theoretical understanding is limited. In this work, we provide evi…
Empirical Study on Optimizer Selection for Out-of-Distribution Generalization
Hiroki Naganuma, Kartik Ahuja, Shiro Takagi +5
Modern deep learning systems do not generalize well when the test data distribution is slightly different to the training data distribution. While much promising work has been acco…
Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation
Divyat Mahajan, Ioannis Mitliagkas, Brady Neal +1
We study the problem of model selection in causal inference, specifically for conditional average treatment effect (CATE) estimation. Unlike machine learning, there is no perfect a…
Towards Out-of-Distribution Adversarial Robustness
Adam Ibrahim, Charles Guille-Escuret, Ioannis Mitliagkas +3
Adversarial robustness continues to be a major challenge for deep learning. A core issue is that robustness to one type of attack often fails to transfer to other attacks. While pr…
CADet: Fully Self-Supervised Out-Of-Distribution Detection With Contrastive Learning
Charles Guille-Escuret, Pau Rodriguez, David Vazquez +2
Handling out-of-distribution (OOD) samples has become a major stake in the real-world deployment of machine learning systems. This work explores the use of self-supervised contrast…
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD
Alireza Mousavi-Hosseini, Sejun Park, Manuela Girotti +2
We study the problem of training a two-layer neural network (NN) of arbitrary width using stochastic gradient descent (SGD) where the input is Gaus…
Optimal transport meets noisy label robust loss and MixUp regularization for domain adaptation
Kilian Fatras, Hiroki Naganuma, Ioannis Mitliagkas
It is common in computer vision to be confronted with domain shift: images which have the same class but different acquisition conditions. In domain adaptation (DA), one wants to c…
Towards efficient representation identification in supervised learning
Kartik Ahuja, Divyat Mahajan, Vasilis Syrgkanis +1
Humans have a remarkable ability to disentangle complex sensory inputs (e.g., image, text) into simple factors of variation (e.g., shape, color) without much supervision. This abil…
Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize
Ryan D'Orazio, Nicolas Loizou, Issam Laradji +1
We investigate the convergence of stochastic mirror descent (SMD) under interpolation in relatively smooth and smooth convex optimization. In relatively smooth convex optimization…
Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization
Kartik Ahuja, Ethan Caballero, Dinghuai Zhang +4
The invariance principle from causality is at the heart of notable approaches such as invariant risk minimization (IRM) that seek to address out-of-distribution (OOD) generalizatio…
Gotta Go Fast When Generating Data with Score-Based Models
Alexia Jolicoeur-Martineau, Ke Li, Rémi Piché-Taillefer +2
Score-based (denoising diffusion) generative models have recently gained a lot of success in generating realistic and diverse data. These approaches define a forward diffusion proc…