2.5k citations · 2.7k across the 13 of their papers we have counts for
8 papers · 1 filter
Amortizing intractable inference in large language models
Edward J. Hu, Moksh Jain, Eric Elmoznino +4
Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This limits tractable querying of this knowled…
GFlowNet-EM for learning compositional latent variable models
Edward J. Hu, Nikolay Malkin, Moksh Jain +3
Latent variable models (LVMs) with discrete compositional latents are an important but challenging setting due to a combinatorially large number of possible configurations of the l…
GFlowNets and variational inference
Esmeralda S. Whitammer, Salem Lahlou, Tristan Deleu +5
This paper builds bridges between two families of probabilistic algorithms: (hierarchical) variational inference (VI), which is typically used to model distributions over continuou…
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
Greg Yang, Edward J. Hu, Igor Babuschkin +7
Hyperparameter (HP) tuning in deep learning is an expensive process, prohibitively so for neural networks (NNs) with billions of parameters. We show that, in the recently discovere…
GFlowNet Foundations
Yoshua Bengio, Salem Lahlou, Tristan Deleu +3
Generative Flow Networks (GFlowNets) have been introduced as a method to sample a diverse set of candidates in an active learning context, with a training objective that makes them…
Feature Learning in Infinite-Width Neural Networks
Greg Yang, Edward J. Hu
As its width tends to infinity, a deep neural network's behavior under gradient descent can become simplified and predictable (e.g. given by the Neural Tangent Kernel (NTK)), if it…
Improved Image Wasserstein Attacks and Defenses
Edward J. Hu, Adith Swaminathan, Hadi Salman +1
Robustness against image perturbations bounded by a ball have been well-studied in recent literature. Perturbations in the real-world, however, rarely exhibit the pixel in…
Randomized Smoothing of All Shapes and Sizes
Greg Yang, Tony Duan, J. Edward Hu +3
Randomized smoothing is the current state-of-the-art defense with provable robustness against adversarial attacks. Many works have devised new randomized smoothing schemes…