565 citations · 3k across the 108 of their papers we have counts for
13 papers · 2 filters
Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training
Bo Zheng, Li Dong, Shaohan Huang +5
Compared to monolingual models, cross-lingual models usually require a more expressive vocabulary to represent all languages adequately. We find that many languages are under-repre…
CausalBERT: Injecting Causal Knowledge Into Pre-trained Models with Minimal Supervision
Zhongyang Li, Xiao Ding, Kuo Liao +2
Recent work has shown success in incorporating pre-trained models like BERT to improve NLP systems. However, existing pre-trained models lack of causal knowledge which prevents tod…
Guided Generation of Cause and Effect
Zhongyang Li, Xiao Ding, Ting Liu +2
We present a conditional text generation framework that posits sentential expressions of possible causes and effects. This framework depends on two novel resources we develop in th…
Consistency Regularization for Cross-Lingual Fine-Tuning
Bo Zheng, Li Dong, Shaohan Huang +7
Fine-tuning pre-trained cross-lingual language models can transfer task-specific supervision from one language to the others. In this work, we propose to improve cross-lingual fine…
BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data
Haoyu Song, Yan Wang, Kaiyan Zhang +2
Maintaining consistent personas is essential for dialogue agents. Although tremendous advancements have been brought, the limited-scale of annotated persona-dense data are still ba…
Adversarial Training for Machine Reading Comprehension with Virtual Embeddings
Ziqing Yang, Yiming Cui, Chenglei Si +4
Adversarial training (AT) as a regularization method has proved its effectiveness on various tasks. Though there are successful applications of AT on some NLP tasks, the distinguis…