2.5k citations · 2.7k across the 13 of their papers we have counts for
14 papers
Amortizing intractable inference in large language models
Edward J. Hu, Moksh Jain, Eric Elmoznino +4
Autoregressive large language models (LLMs) compress knowledge from their training data through next-token conditional distributions. This limits tractable querying of this knowled…
Differentiable Tree Operations Promote Compositional Generalization
Paul Soulos, Edward Hu, Kate McCurdy +4
In the context of structure-to-structure transformation tasks, learning sequences of discrete symbolic operations poses significant challenges due to their non-differentiability. T…
GFlowNet-EM for learning compositional latent variable models
Edward J. Hu, Nikolay Malkin, Moksh Jain +3
Latent variable models (LVMs) with discrete compositional latents are an important but challenging setting due to a combinatorially large number of possible configurations of the l…
GFlowNets and variational inference
Esmeralda S. Whitammer, Salem Lahlou, Tristan Deleu +5
This paper builds bridges between two families of probabilistic algorithms: (hierarchical) variational inference (VI), which is typically used to model distributions over continuou…
Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
Greg Yang, Edward J. Hu, Igor Babuschkin +7
Hyperparameter (HP) tuning in deep learning is an expensive process, prohibitively so for neural networks (NNs) with billions of parameters. We show that, in the recently discovere…
GFlowNet Foundations
Yoshua Bengio, Salem Lahlou, Tristan Deleu +3
Generative Flow Networks (GFlowNets) have been introduced as a method to sample a diverse set of candidates in an active learning context, with a training objective that makes them…
Guided Generation of Cause and Effect
Zhongyang Li, Xiao Ding, Ting Liu +2
We present a conditional text generation framework that posits sentential expressions of possible causes and effects. This framework depends on two novel resources we develop in th…
LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu, Yelong Shen, Phillip Wallis +5
An important paradigm of natural language processing consists of large-scale pre-training on general domain data and adaptation to particular tasks or domains. As we pre-train larg…
Feature Learning in Infinite-Width Neural Networks
Greg Yang, Edward J. Hu
As its width tends to infinity, a deep neural network's behavior under gradient descent can become simplified and predictable (e.g. given by the Neural Tangent Kernel (NTK)), if it…
Iterative Paraphrastic Augmentation with Discriminative Span Alignment
Ryan Culkin, J. Edward Hu, Elias Stengel-Eskin +2
We introduce a novel paraphrastic augmentation strategy based on sentence-level lexically constrained paraphrasing and discriminative span alignment. Our approach allows for the la…
Improved Image Wasserstein Attacks and Defenses
Edward J. Hu, Adith Swaminathan, Hadi Salman +1
Robustness against image perturbations bounded by a ball have been well-studied in recent literature. Perturbations in the real-world, however, rarely exhibit the pixel in…
Randomized Smoothing of All Shapes and Sizes
Greg Yang, Tony Duan, J. Edward Hu +3
Randomized smoothing is the current state-of-the-art defense with provable robustness against adversarial attacks. Many works have devised new randomized smoothing schemes…
ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation
J. Edward Hu, Rachel Rudinger, Matt Post +1
We present ParaBank, a large-scale English paraphrase dataset that surpasses prior work in both quantity and quality. Following the approach of ParaNMT, we train a Czech-English ne…
Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation
Adam Poliak, Aparajita Haldar, Rachel Rudinger +4
We present a large-scale collection of diverse natural language inference (NLI) datasets that help provide insight into how well a sentence representation captures distinct types o…