1.8k citations · 4.6k across the 52 of their papers we have counts for
8 papers · 2 filters
On the Sentence Embeddings from Pre-trained Language Models
Bohan Li, Hao Zhou, Junxian He +3
Pre-trained contextual representations like BERT have achieved great success in natural language processing. However, the sentence embeddings from the pre-trained language models w…
EIGEN: Event Influence GENeration using Pre-trained Language Models
Aman Madaan, Dheeraj Rajagopal, Yiming Yang +3
Reasoning about events and tracking their influences is fundamental to understanding processes. In this paper, we present EIGEN - a method to leverage pre-trained language models t…
JAKET: Joint Pre-training of Knowledge Graph and Language Understanding
Donghan Yu, Chenguang Zhu, Yiming Yang +1
Knowledge graphs (KGs) contain rich information about world knowledge, entities and relations. Thus, they can be great supplements to existing pre-trained language models. However,…
Unsupervised Parallel Corpus Mining on Web Data
Guokun Lai, Zihang Dai, Yiming Yang
With a large amount of parallel data, neural machine translation systems are able to deliver human-level performance for sentence-level translation. However, it is costly to label…
Predicting Performance for Natural Language Processing Tasks
Mengzhou Xia, Antonios Anastasopoulos, Ruochen Xu +2
Given the complexity of combinations of tasks, languages, and domains in natural language processing (NLP) research, it is computationally prohibitive to exhaustively test newly pr…
Politeness Transfer: A Tag and Generate Approach
Aman Madaan, Amrith Setlur, Tanmay Parekh +6
This paper introduces a new task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning. We also provide a dataset o…
Practical Comparable Data Collection for Low-Resource Languages via Images
Aman Madaan, Shruti Rijhwani, Antonios Anastasopoulos +2
We propose a method of curating high-quality comparable training data for low-resource languages with monolingual annotators. Our method involves using a carefully selected set of…
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Zhiqing Sun, Hongkun Yu, Xiaodan Song +3
Natural Language Processing (NLP) has recently achieved great success by using huge pre-trained models with hundreds of millions of parameters. However, these models suffer from he…