1.1k citations · 2.8k across the 21 of their papers we have counts for
23 papers
Entailment as Robust Self-Learner
Jiaxin Ge, Hongyin Luo, Yoon Kim +1
Entailment has been recognized as an important metric for evaluating natural language understanding (NLU) models, and recent studies have found that entailment pretraining benefits…
Deriving Language Models from Masked Language Models
Lucas Torroba Hennigen, Yoon Kim
Masked language models (MLM) do not explicitly define a distribution over language, i.e., they are not language models per se. However, recent work has implicitly treated them as s…
Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach
Han Guo, Philip Greengard, Hongyi Wang +3
The canonical formulation of federated learning treats it as a distributed optimization problem where the model parameters are optimized against a global loss function that decompo…
Large Language Models are Few-Shot Clinical Information Extractors
Monica Agrawal, Stefan Hegselmann, Hunter Lang +2
A long-running goal of the clinical NLP community is the extraction of important variables trapped in clinical notes. However, roadblocks have included dataset shift from the gener…
Controlling the Focus of Pretrained Language Generation Models
Jiabao Ji, Yoon Kim, James Glass +1
The finetuning of pretrained transformer-based language generation models are typically conducted in an end-to-end manner, where the model learns to attend to relevant parts of the…
Co-training Improves Prompt-based Learning for Large Language Models
Hunter Lang, Monica Agrawal, Yoon Kim +1
We demonstrate that co-training (Blum & Mitchell, 1998) can improve the performance of prompt-based learning by using unlabeled data. While prompting has emerged as a promising par…
Compound Probabilistic Context-Free Grammars for Grammar Induction
Yoon Kim, Chris Dyer, Alexander M. Rush
We study a formalization of the grammar induction problem that models sentences as being generated by a compound probabilistic context-free grammar. In contrast to traditional form…
Amortized Bethe Free Energy Minimization for Learning MRFs
Sam Wiseman, Yoon Kim
We propose to learn deep undirected graphical models (i.e., MRFs) with a non-ELBO objective for which we can calculate exact gradients. In particular, we optimize a saddle-point ob…
Unsupervised Recurrent Neural Network Grammars
Yoon Kim, Alexander M. Rush, Lei Yu +3
Recurrent neural network grammars (RNNG) are generative models of language which jointly model syntax and surface structure by incrementally generating a syntax tree and sentence i…
A Tutorial on Deep Latent Variable Models of Natural Language
Yoon Kim, Sam Wiseman, Alexander M. Rush
There has been much recent, exciting work on combining the complementary strengths of latent variable models and deep learning. Latent variable modeling makes it easy to explicitly…
Avoiding Latent Variable Collapse With Generative Skip Models
Adji B. Dieng, Yoon Kim, Alexander M. Rush +1
Variational autoencoders learn distributions of high-dimensional data. They model data with a deep latent-variable model and then fit the model by maximizing a lower bound of the l…
Latent Alignment and Variational Attention
Yuntian Deng, Yoon Kim, Justin Chiu +2
Neural attention has become central to many state-of-the-art models in natural language processing and related domains. Attention networks are an easy-to-train and effective method…
OpenNMT: Neural Machine Translation Toolkit
Guillaume Klein, Yoon Kim, Yuntian Deng +3
OpenNMT is an open-source toolkit for neural machine translation (NMT). The system prioritizes efficiency, modularity, and extensibility with the goal of supporting NMT research in…
Semi-Amortized Variational Autoencoders
Yoon Kim, Sam Wiseman, Andrew C. Miller +2
Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network. While AVI has enabled efficient training of deep generative models…
Adapting Sequence Models for Sentence Correction
Allen Schmaltz, Yoon Kim, Alexander M. Rush +1
In a controlled experiment of sequence-to-sequence approaches for the task of sentence correction, we find that character-based models are generally more effective than word-based…
Adversarially Regularized Autoencoders
Jake Zhao, Yoon Kim, Kelly Zhang +2
Deep latent variable models, trained using variational autoencoders or generative adversarial networks, are now a key technique for representation learning of continuous structures…
Structured Attention Networks
Yoon Kim, Carl Denton, Luong Hoang +1
Attention networks have proven to be an effective approach for embedding categorical inference within a deep neural network. However, for many tasks we may want to model richer str…
Sequence-Level Knowledge Distillation
Yoon Kim, Alexander M. Rush
Neural machine translation (NMT) offers a novel alternative formulation of translation that is potentially simpler than statistical approaches. However to reach competitive perform…
Sentence-Level Grammatical Error Identification as Sequence-to-Sequence Correction
Allen Schmaltz, Yoon Kim, Alexander M. Rush +1
We demonstrate that an attention-based encoder-decoder model can be used for sentence-level grammatical error identification for the Automated Evaluation of Scientific Writing (AES…
Character-Aware Neural Language Models
Yoon Kim, Yacine Jernite, David Sontag +1
We describe a simple neural language model that relies only on character-level inputs. Predictions are still made at the word-level. Our model employs a convolutional neural networ…
Convolutional Neural Networks for Sentence Classification
Yoon Kim
We report on a series of experiments with convolutional neural networks (CNN) trained on top of pre-trained word vectors for sentence-level classification tasks. We show that a sim…
Credibility Adjusted Term Frequency: A Supervised Term Weighting Scheme for Sentiment Analysis and Text Classification
Yoon Kim, Owen Zhang
We provide a simple but novel supervised weighting scheme for adjusting term frequency in tf-idf for sentiment analysis and text classification. We compare our method to baseline w…
Temporal Analysis of Language through Neural Language Models
Yoon Kim, Yi-I Chiu, Kentaro Hanaki +2
We provide a method for automatically detecting change in language across time through a chronologically trained neural language model. We train the model on the Google Books Ngram…