142 citations · 226 across the 24 of their papers we have counts for
14 papers · 1 filter
Universal interpolation for deep residual self-attention networks
Sibylle Marcotte, Joan Bruna
Universal approximation is a necessary qualitative property of learning architectures to benefit from scaling laws. While it is generically verified on a variety of neural architec…
Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitation
David Brandfonbrener, Ofir Nachum, Joan Bruna
In recent years, domains such as natural language processing and image recognition have popularized the paradigm of using large datasets to pretrain representations that can be eff…
Beyond the Edge of Stability via Two-step Gradient Updates
Lei Chen, Joan Bruna
Gradient Descent (GD) is a powerful workhorse of modern machine learning thanks to its scalability and efficiency in high-dimensional spaces. Its ability to find local minimisers i…
When does return-conditioned supervised learning work for offline reinforcement learning?
David Brandfonbrener, Alberto Bietti, Jacob Buckman +2
Several recent works have proposed a class of algorithms for the offline reinforcement learning (RL) problem that we will refer to as return-conditioned supervised learning (RCSL).…
Quantile Filtered Imitation Learning
David Brandfonbrener, William F. Whitney, Rajesh Ranganath +1
We introduce quantile filtered imitation learning (QFIL), a novel policy improvement operator designed for offline reinforcement learning. QFIL performs policy improvement by runni…
Offline RL Without Off-Policy Evaluation
David Brandfonbrener, William F. Whitney, Rajesh Ranganath +1
Most prior approaches to offline reinforcement learning (RL) have taken an iterative actor-critic approach involving off-policy evaluation. In this paper we show that simply doing…
On Energy-Based Models with Overparametrized Shallow Neural Networks
Carles Domingo-Enrich, Alberto Bietti, Eric Vanden-Eijnden +1
Energy-based models (EBMs) are a simple yet powerful framework for generative modeling. They are based on a trainable energy function which defines an associated Gibbs measure, and…
Depth separation beyond radial functions
Luca Venturi, Samy Jelassi, Tristan Ozuch +1
High-dimensional depth separation results for neural networks show that certain functions can be efficiently approximated by two-hidden-layer networks but not by one-hidden-layer o…
On Graph Neural Networks versus Graph-Augmented MLPs
Lei Chen, Zhengdao Chen, Joan Bruna
From the perspective of expressive power, this work compares multi-layer Graph Neural Networks (GNNs) with a simplified alternative that we call Graph-Augmented Multi-Layer Percept…
Kernel-Based Smoothness Analysis of Residual Networks
Tom Tirer, Joan Bruna, Raja Giryes
A major factor in the success of deep neural networks is the use of sophisticated architectures rather than the classical multilayer perceptron (MLP). Residual networks (ResNets) s…
A Functional Perspective on Learning Symmetric Functions with Neural Networks
Aaron Zweig, Joan Bruna
Symmetric functions, which take as input an unordered, fixed-size set, are known to be universally representable by neural networks that enforce permutation invariance. These archi…
In-Distribution Interpretability for Challenging Modalities
Cosmas Heiß, Ron Levie, Cinjon Resnick +2
It is widely recognized that the predictions of deep neural networks are difficult to parse relative to simpler approaches. However, the development of methods to investigate the m…
On Sparsity in Overparametrised Shallow ReLU Networks
Jaume de Dios, Joan Bruna
The analysis of neural network training beyond their linearization regime remains an outstanding open question, even in the simplest setup of a single hidden-layer. The limit of in…
Offline Contextual Bandits with Overparameterized Models
David Brandfonbrener, William F. Whitney, Rajesh Ranganath +1
Recent results in supervised learning suggest that while overparameterized models have the capacity to overfit, they in fact generalize quite well. We ask whether the same phenomen…