83 citations · 264 across the 31 of their papers we have counts for
11 papers · 1 filter
TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning
Yixuan Su, Fangyu Liu, Zaiqiao Meng +4
Masked language models (MLMs) such as BERT and RoBERTa have revolutionized the field of Natural Language Understanding in the past few years. However, existing pre-trained MLMs oft…
Visually Grounded Reasoning across Languages and Cultures
Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti +3
The design of widespread vision-and-language datasets and pre-trained encoders directly adopts, or draws inspiration from, the concepts and images of ImageNet. While one can hardly…
Prix-LM: Pretraining for Multilingual Knowledge Base Construction
Wenxuan Zhou, Fangyu Liu, Ivan Vulić +2
Knowledge bases (KBs) contain plenty of structured world and commonsense knowledge. As such, they often complement distributional text-based information and facilitate various down…
Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pre-trained Language Models
Zaiqiao Meng, Fangyu Liu, Ehsan Shareghi +3
Knowledge probing is crucial for understanding the knowledge transfer mechanism behind the pre-trained language models (PLMs). Despite the growing progress of probing knowledge for…
MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models
Qianchu Liu, Fangyu Liu, Nigel Collier +2
Recent work indicated that pretrained language models (PLMs) such as BERT and RoBERTa can be transformed into effective sentence and word encoders even via simple self-supervised t…
Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT
Zaiqiao Meng, Fangyu Liu, Thomas Hikaru Clark +2
Infusing factual knowledge into pre-trained models is fundamental for many knowledge-intensive tasks. In this paper, we proposed Mixture-of-Partitions (MoP), an infusion approach t…
Plan-then-Generate: Controlled Data-to-Text Generation via Planning
Yixuan Su, David Vandyke, Sihui Wang +2
Recent developments in neural networks have led to the advance in data-to-text generation. However, the lack of ability of neural models to control the structure of generated outpu…
Few-Shot Table-to-Text Generation with Prototype Memory
Yixuan Su, Zaiqiao Meng, Simon Baker +1
Neural table-to-text generation models have achieved remarkable progress on an array of tasks. However, due to the data-hungry nature of neural models, their performances strongly…
Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking
Fangyu Liu, Ivan Vulić, Anna Korhonen +1
Injecting external domain-specific knowledge (e.g., UMLS) into pretrained language models (LMs) advances their capability to handle specialised in-domain tasks such as biomedical e…
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders
Fangyu Liu, Ivan Vulić, Anna Korhonen +1
Pretrained Masked Language Models (MLMs) have revolutionised NLP in recent years. However, previous work has indicated that off-the-shelf MLMs are not effective as universal lexica…
Non-Autoregressive Text Generation with Pre-trained Language Models
Yixuan Su, Deng Cai, Yan Wang +4
Non-autoregressive generation (NAG) has recently attracted great attention due to its fast inference speed. However, the generation quality of existing NAG models still lags behind…