59 citations · 238 across the 20 of their papers we have counts for
7 papers · 1 filter
Knowledge Enhanced Pretrained Language Models: A Compreshensive Survey
Xiaokai Wei, Shen Wang, Dejiao Zhang +2
Pretrained Language Models (PLM) have established a new paradigm through learning informative contextualized representations on large-scale text corpus. This new paradigm has revol…
Virtual Augmentation Supported Contrastive Learning of Sentence Representations
Dejiao Zhang, Wei Xiao, Henghui Zhu +2
Despite profound successes, contrastive representation learning relies on carefully designed data augmentations using domain specific knowledge. This challenge is magnified in natu…
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora
Xisen Jin, Dejiao Zhang, Henghui Zhu +5
Pretrained language models (PTLMs) are typically learned over a large, static corpus and further fine-tuned for various downstream tasks. However, when deployed in the real world,…
Pairwise Supervised Contrastive Learning of Sentence Representations
Dejiao Zhang, Shang-Wen Li, Wei Xiao +4
Many recent successes in sentence representation learning have been achieved by simply fine-tuning on the Natural Language Inference (NLI) datasets with triplet loss or siamese los…
Improving Factual Consistency of Abstractive Summarization via Question Answering
Feng Nan, Cicero Nogueira dos Santos, Henghui Zhu +7
A commonly observed problem with the state-of-the art abstractive summarization models is that the generated summaries can be factually inconsistent with the input documents. The f…
Supporting Clustering with Contrastive Learning
Dejiao Zhang, Feng Nan, Xiaokai Wei +6
Unsupervised clustering aims at discovering the semantic categories of data according to some distance measured in the representation space. However, different categories often ove…
Entity-level Factual Consistency of Abstractive Text Summarization
Feng Nan, Ramesh Nallapati, Zhiguo Wang +5
A key challenge for abstractive summarization is ensuring factual consistency of the generated summary with respect to the original document. For example, state-of-the-art models t…