24 citations · 52 across the 16 of their papers we have counts for
17 papers
Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning
Yatin Dandi, Matteo Vilucchio, Luca Arnaboldi +2
Understanding how deep neural networks learn useful internal representations from data remains a central open problem in the theory of deep learning. We introduce Neural Low-Degree…
Fundamental computational limits of weak learnability in high-dimensional multi-index models
Emanuele Troiani, Yatin Dandi, Leonardo Defilippis +3
Multi-index models - functions which only depend on the covariates through a non-linear transformation of their projection on a subspace - are a useful benchmark for investigating…
Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions
Luca Arnaboldi, Yatin Dandi, Florent Krzakala +2
Neural networks can identify low-dimensional relevant structures within high-dimensional noisy data, yet our mathematical understanding of how they do so remains scarce. Here, we i…
Asymptotics of feature learning in two-layer networks after one gradient-step
Hugo Cui, Luca Pesce, Yatin Dandi +4
In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single grad…
The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents
Yatin Dandi, Emanuele Troiani, Luca Arnaboldi +3
We investigate the training dynamics of two-layer neural networks when learning multi-index target functions. We focus on multi-pass gradient descent (GD) that reuses the batches m…
A Gentle Introduction to Gradient-Based Optimization and Variational Inequalities for Machine Learning
Neha S. Wadia, Yatin Dandi, Michael I. Jordan
The rapid progress in machine learning in recent years has been based on a highly productive connection to gradient-based optimization. Further progress hinges in part on a shift i…
Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective
Davide Ghio, Yatin Dandi, Florent Krzakala +1
Recent years witnessed the development of powerful generative models based on flows, diffusion or autoregressive neural networks, achieving remarkable success in generating data fr…
How Two-Layer Neural Networks Learn, One (Giant) Step at a Time
Yatin Dandi, Florent Krzakala, Bruno Loureiro +2
For high-dimensional Gaussian data, we investigate theoretically how the features of a two-layer neural network adapt to the structure of the target function through a few large ba…
Maximally-stable Local Optima in Random Graphs and Spin Glasses: Phase Transitions and Universality
Yatin Dandi, David Gamarnik, Lenka Zdeborová
We consider -stable local optima of Ising spin glass models, defined as spin configurations such that for nearly all of the spins, flipping their values results in increasing en…
Universality laws for Gaussian mixtures in generalized linear models
Yatin Dandi, Ludovic Stephan, Florent Krzakala +2
Let denote independent samples from a general mixture distribution , and consider the hypothesis class of gener…
Data-heterogeneity-aware Mixing for Decentralized Learning
Yatin Dandi, Anastasia Koloskova, Martin Jaggi +1
Decentralized learning provides an effective framework to train machine learning models with data distributed over arbitrary communication graphs. However, most existing approaches…
NeurInt : Learning to Interpolate through Neural ODEs
Avinandan Bose, Aniket Das, Yatin Dandi +1
A wide range of applications require learning image generation models whose latent space effectively captures the high-level factors of variation present in the data distribution.…
Understanding Layer-wise Contributions in Deep Neural Networks through Spectral Analysis
Yatin Dandi, Arthur Jacot
Spectral analysis is a powerful tool, decomposing any function into simpler parts. In machine learning, Mercer's theorem generalizes this idea, providing for any kernel and input d…
Implicit Gradient Alignment in Distributed and Federated Learning
Yatin Dandi, Luis Barba, Martin Jaggi
A major obstacle to achieving global convergence in distributed and federated learning is the misalignment of gradients across clients, or mini-batches due to heterogeneity and sto…
Model-Agnostic Learning to Meta-Learn
Arnout Devos, Yatin Dandi
In this paper, we propose a learning algorithm that enables a model to quickly exploit commonalities among related tasks from an unseen task distribution, before quickly adapting t…
Generalized Adversarially Learned Inference
Yatin Dandi, Homanga Bharadhwaj, Abhishek Kumar +1
Allowing effective inference of latent vectors while training GANs can greatly increase their applicability in various downstream tasks. Recent approaches, such as ALI and BiGAN fr…
Jointly Trained Image and Video Generation using Residual Vectors
Yatin Dandi, Aniket Das, Soumye Singhal +2
In this work, we propose a modeling technique for jointly training image and video generation models by simultaneously learning to map latent variables with a fixed prior onto real…