773 citations · 1.4k across the 26 of their papers we have counts for
6 papers · 1 filter
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer
Siyu Ding, Junyuan Shang, Shuohuan Wang +4
Transformers are not suited for processing long documents, due to their quadratically increasing memory and time consumption. Simply truncating a long document or applying the spar…
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora
Xuan Ouyang, Shuohuan Wang, Chao Pang +4
Recent studies have demonstrated that pre-trained cross-lingual models achieve impressive performance in downstream cross-lingual tasks. This improvement benefits from learning a l…
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding
Dongling Xiao, Yu-Kun Li, Han Zhang +4
Coarse-grained linguistic information, such as named entities or phrases, facilitates adequately representation learning in pre-training. Previous works mainly focus on extending t…
ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph
Fei Yu, Jiji Tang, Weichong Yin +4
We propose a knowledge-enhanced approach, ERNIE-ViL, which incorporates structured knowledge obtained from scene graphs to learn joint representations of vision-language. ERNIE-ViL…
SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis
Hao Tian, Can Gao, Xinyan Xiao +5
Recently, sentiment analysis has seen remarkable advance with the help of pre-training approaches. However, sentiment knowledge, such as sentiment words and aspect-sentiment pairs,…
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation
Dongling Xiao, Han Zhang, Yukun Li +4
Current pre-training works in natural language generation pay little attention to the problem of exposure bias on downstream tasks. To address this issue, we propose an enhanced mu…