1.8k citations · 4.6k across the 52 of their papers we have counts for
15 papers · 1 filter
On the Sentence Embeddings from Pre-trained Language Models
Bohan Li, Hao Zhou, Junxian He +3
Pre-trained contextual representations like BERT have achieved great success in natural language processing. However, the sentence embeddings from the pre-trained language models w…
Rethinking Transformer-based Set Prediction for Object Detection
Zhiqing Sun, Shengcao Cao, Yiming Yang +1
DETR is a recently proposed Transformer-based method which views object detection as a set prediction problem and achieves state-of-the-art performance but demands extra-long train…
EIGEN: Event Influence GENeration using Pre-trained Language Models
Aman Madaan, Dheeraj Rajagopal, Yiming Yang +3
Reasoning about events and tracking their influences is fundamental to understanding processes. In this paper, we present EIGEN - a method to leverage pre-trained language models t…
JAKET: Joint Pre-training of Knowledge Graph and Language Understanding
Donghan Yu, Chenguang Zhu, Yiming Yang +1
Knowledge graphs (KGs) contain rich information about world knowledge, entities and relations. Thus, they can be great supplements to existing pre-trained language models. However,…
Unsupervised Parallel Corpus Mining on Web Data
Guokun Lai, Zihang Dai, Yiming Yang
With a large amount of parallel data, neural machine translation systems are able to deliver human-level performance for sentence-level translation. However, it is costly to label…
An EM Approach to Non-autoregressive Conditional Sequence Generation
Zhiqing Sun, Yiming Yang
Autoregressive (AR) models have been the dominating approach to conditional sequence generation, but are suffering from the issue of high inference latency. Non-autoregressive (NAR…
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing
Zihang Dai, Guokun Lai, Yiming Yang +1
With the success of language pretraining, it is highly desirable to develop more efficient architectures of good scalability that can exploit the abundant unlabeled data at a lower…
Knowledge Embedding Based Graph Convolutional Network
Donghan Yu, Yiming Yang, Ruohong Zhang +1
Recently, a considerable literature has grown up around the theme of Graph Convolutional Network (GCN). How to effectively leverage the rich structural information in complex graph…
Predicting Performance for Natural Language Processing Tasks
Mengzhou Xia, Antonios Anastasopoulos, Ruochen Xu +2
Given the complexity of combinations of tasks, languages, and domains in natural language processing (NLP) research, it is computationally prohibitive to exhaustively test newly pr…
Politeness Transfer: A Tag and Generate Approach
Aman Madaan, Amrith Setlur, Tanmay Parekh +6
This paper introduces a new task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning. We also provide a dataset o…
Practical Comparable Data Collection for Low-Resource Languages via Images
Aman Madaan, Shruti Rijhwani, Antonios Anastasopoulos +2
We propose a method of curating high-quality comparable training data for low-resource languages with monolingual annotators. Our method involves using a carefully selected set of…
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Zhiqing Sun, Hongkun Yu, Xiaodan Song +3
Natural Language Processing (NLP) has recently achieved great success by using huge pre-trained models with hundreds of millions of parameters. However, these models suffer from he…
Correlation-aware Unsupervised Change-point Detection via Graph Neural Networks
Ruohong Zhang, Yu Hao, Donghan Yu +3
Change-point detection (CPD) aims to detect abrupt changes over time series data. Intuitively, effective CPD over multivariate time series should require explicit modeling of the d…
VIOLIN: A Large-Scale Dataset for Video-and-Language Inference
Jingzhou Liu, Wenhu Chen, Yu Cheng +4
We introduce a new task, Video-and-Language Inference, for joint multimodal understanding of video and text. Given a video clip with aligned subtitles as premise, paired with a nat…
Pre-training Tasks for Embedding-based Large-scale Retrieval
Wei-Cheng Chang, Felix X. Yu, Yin-Wen Chang +2
We consider the large-scale query-document retrieval problem: given a query (e.g., a question), return the set of relevant documents (e.g., paragraphs containing the answer) from a…