11 citations · 54 across the 46 of their papers we have counts for
7 papers · 1 filter
TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning
Yixuan Su, Fangyu Liu, Zaiqiao Meng +4
Masked language models (MLMs) such as BERT and RoBERTa have revolutionized the field of Natural Language Understanding in the past few years. However, existing pre-trained MLMs oft…
The Neglected Sibling: Isotropic Gaussian Posterior for VAE
Lan Zhang, Wray Buntine, Ehsan Shareghi
Deep generative models have been widely used in several areas of NLP, and various techniques have been proposed to augment them or address their training challenges. In this paper,…
It is Not as Good as You Think! Evaluating Simultaneous Machine Translation on Interpretation Data
Jinming Zhao, Philip Arthur, Gholamreza Haffari +2
Most existing simultaneous machine translation (SiMT) systems are trained and evaluated on offline translation corpora. We argue that SiMT systems should be trained and tested on r…
Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pre-trained Language Models
Zaiqiao Meng, Fangyu Liu, Ehsan Shareghi +3
Knowledge probing is crucial for understanding the knowledge transfer mechanism behind the pre-trained language models (PLMs). Despite the growing progress of probing knowledge for…
Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT
Zaiqiao Meng, Fangyu Liu, Thomas Hikaru Clark +2
Infusing factual knowledge into pre-trained models is fundamental for many knowledge-intensive tasks. In this paper, we proposed Mixture-of-Partitions (MoP), an infusion approach t…
Unsupervised Representation Disentanglement of Text: An Evaluation on Synthetic Datasets
Lan Zhang, Victor Prokhorov, Ehsan Shareghi
To highlight the challenges of achieving representation disentanglement for text domain in an unsupervised setting, in this paper we select a representative set of successfully app…
Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification
Yi Zhu, Ehsan Shareghi, Yingzhen Li +2
Semi-supervised learning through deep generative models and multi-lingual pretraining techniques have orchestrated tremendous success across different areas of NLP. Nonetheless, th…