142 citations · 226 across the 24 of their papers we have counts for
26 papers
Universal interpolation for deep residual self-attention networks
Sibylle Marcotte, Joan Bruna
Universal approximation is a necessary qualitative property of learning architectures to benefit from scaling laws. While it is generically verified on a variety of neural architec…
Reliable coarse-grained turbulent simulations through combined offline learning and neural emulation
Christian Pedersen, Laure Zanna, Joan Bruna +1
Integration of machine learning (ML) models of unresolved dynamics into numerical simulations of fluid dynamics has been demonstrated to improve the accuracy of coarse resolution s…
Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitation
David Brandfonbrener, Ofir Nachum, Joan Bruna
In recent years, domains such as natural language processing and image recognition have popularized the paradigm of using large datasets to pretrain representations that can be eff…
Data-driven multiscale modeling for correcting dynamical systems
Karl Otness, Laure Zanna, Joan Bruna
We propose a multiscale approach for predicting quantities in dynamical systems which is explicitly structured to extract information in both fine-to-coarse and coarse-to-fine dire…
Beyond the Edge of Stability via Two-step Gradient Updates
Lei Chen, Joan Bruna
Gradient Descent (GD) is a powerful workhorse of modern machine learning thanks to its scalability and efficiency in high-dimensional spaces. Its ability to find local minimisers i…
When does return-conditioned supervised learning work for offline reinforcement learning?
David Brandfonbrener, Alberto Bietti, Jacob Buckman +2
Several recent works have proposed a class of algorithms for the offline reinforcement learning (RL) problem that we will refer to as return-conditioned supervised learning (RCSL).…
Quantile Filtered Imitation Learning
David Brandfonbrener, William F. Whitney, Rajesh Ranganath +1
We introduce quantile filtered imitation learning (QFIL), a novel policy improvement operator designed for offline reinforcement learning. QFIL performs policy improvement by runni…
Offline RL Without Off-Policy Evaluation
David Brandfonbrener, William F. Whitney, Rajesh Ranganath +1
Most prior approaches to offline reinforcement learning (RL) have taken an iterative actor-critic approach involving off-policy evaluation. In this paper we show that simply doing…
On Energy-Based Models with Overparametrized Shallow Neural Networks
Carles Domingo-Enrich, Alberto Bietti, Eric Vanden-Eijnden +1
Energy-based models (EBMs) are a simple yet powerful framework for generative modeling. They are based on a trainable energy function which defines an associated Gibbs measure, and…
Symmetry Breaking in Symmetric Tensor Decomposition
Yossi Arjevani, Joan Bruna, Michael Field +3
In this note, we consider the highly nonconvex optimization problem associated with computing the rank decomposition of symmetric tensors. We formulate the invariance properties of…
Depth separation beyond radial functions
Luca Venturi, Samy Jelassi, Tristan Ozuch +1
High-dimensional depth separation results for neural networks show that certain functions can be efficiently approximated by two-hidden-layer networks but not by one-hidden-layer o…
Self-Supervised Equivariant Scene Synthesis from Video
Cinjon Resnick, Or Litany, Cosmas Heiß +3
We propose a self-supervised framework to learn scene representations from video that are automatically delineated into background, characters, and their animations. Our method cap…
On Graph Neural Networks versus Graph-Augmented MLPs
Lei Chen, Zhengdao Chen, Joan Bruna
From the perspective of expressive power, this work compares multi-layer Graph Neural Networks (GNNs) with a simplified alternative that we call Graph-Augmented Multi-Layer Percept…
Learned Equivariant Rendering without Transformation Supervision
Cinjon Resnick, Or Litany, Hugo Larochelle +2
We propose a self-supervised framework to learn scene representations from video that are automatically delineated into objects and background. Our method relies on moving objects…
Adaptive Test Allocation for Outbreak Detection and Tracking in Social Contact Networks
Pau Batlle, Joan Bruna, Carlos Fernandez-Granda +1
We present a general framework for adaptive allocation of viral tests in social contact networks. We pose and solve several complementary problems. First, we consider the design of…
Kernel-Based Smoothness Analysis of Residual Networks
Tom Tirer, Joan Bruna, Raja Giryes
A major factor in the success of deep neural networks is the use of sophisticated architectures rather than the classical multilayer perceptron (MLP). Residual networks (ResNets) s…
A Dynamical Central Limit Theorem for Shallow Neural Networks
Zhengdao Chen, Grant M. Rotskoff, Joan Bruna +1
Recent theoretical works have characterized the dynamics of wide shallow neural networks trained via gradient descent in an asymptotic mean-field limit when the width tends towards…
A Functional Perspective on Learning Symmetric Functions with Neural Networks
Aaron Zweig, Joan Bruna
Symmetric functions, which take as input an unordered, fixed-size set, are known to be universally representable by neural networks that enforce permutation invariance. These archi…
Depth separation for reduced deep networks in nonlinear model reduction: Distilling shock waves in nonlinear hyperbolic problems
Donsub Rim, Luca Venturi, Joan Bruna +1
Classical reduced models are low-rank approximations using a fixed basis designed to achieve dimensionality reduction of large-scale systems. In this work, we introduce reduced dee…
In-Distribution Interpretability for Challenging Modalities
Cosmas Heiß, Ron Levie, Cinjon Resnick +2
It is widely recognized that the predictions of deep neural networks are difficult to parse relative to simpler approaches. However, the development of methods to investigate the m…
On Sparsity in Overparametrised Shallow ReLU Networks
Jaume de Dios, Joan Bruna
The analysis of neural network training beyond their linearization regime remains an outstanding open question, even in the simplest setup of a single hidden-layer. The limit of in…
IDEAL: Inexact DEcentralized Accelerated Augmented Lagrangian Method
Yossi Arjevani, Joan Bruna, Bugra Can +3
We introduce a framework for designing primal methods under the decentralized optimization setting where local functions are smooth and strongly convex. Our approach consists of ap…
Offline Contextual Bandits with Overparameterized Models
David Brandfonbrener, William F. Whitney, Rajesh Ranganath +1
Recent results in supervised learning suggest that while overparameterized models have the capacity to overfit, they in fact generalize quite well. We ask whether the same phenomen…
Neural Splines: Fitting 3D Surfaces with Infinitely-Wide Neural Networks
Francis Williams, Matthew Trager, Joan Bruna +1
We present Neural Splines, a technique for 3D surface reconstruction that is based on random feature kernels arising from infinitely-wide shallow ReLU networks. Our method achieves…