38 citations · 238 across the 42 of their papers we have counts for
9 papers · 1 filter
Measuring Context-Word Biases in Lexical Semantic Datasets
Qianchu Liu, Diana McCarthy, Anna Korhonen
State-of-the-art pretrained contextualized models (PCM) eg. BERT use tasks such as WiC and WSD to evaluate their word-in-context representations. This inherently assumes that perfo…
Composable Sparse Fine-Tuning for Cross-Lingual Transfer
Alan Ansell, Edoardo Maria Ponti, Anna Korhonen +1
Fine-tuning the entire set of parameters of a large pretrained model has become the mainstream approach for transfer learning. To increase its efficiency and prevent catastrophic f…
MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models
Qianchu Liu, Fangyu Liu, Nigel Collier +2
Recent work indicated that pretrained language models (PLMs) such as BERT and RoBERTa can be transformed into effective sentence and word encoders even via simple self-supervised t…
Towards Zero-shot Language Modeling
Edoardo Maria Ponti, Ivan Vulić, Ryan Cotterell +2
Can we construct a neural model that is inductively biased towards learning human languages? Motivated by this question, we aim at constructing an informative prior over neural wei…
Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking
Fangyu Liu, Ivan Vulić, Anna Korhonen +1
Injecting external domain-specific knowledge (e.g., UMLS) into pretrained language models (LMs) advances their capability to handle specialised in-domain tasks such as biomedical e…
AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples
Qianchu Liu, Edoardo M. Ponti, Diana McCarthy +2
Capturing word meaning in context and distinguishing between correspondences and variations across languages is key to building successful multilingual and cross-lingual text repre…
Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems
Evgeniia Razumovskaia, Goran Glavaš, Olga Majewska +3
In task-oriented dialogue (ToD), a user holds a conversation with an artificial agent to complete a concrete task. Although this technology represents one of the central objectives…
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders
Fangyu Liu, Ivan Vulić, Anna Korhonen +1
Pretrained Masked Language Models (MLMs) have revolutionised NLP in recent years. However, previous work has indicated that off-the-shelf MLMs are not effective as universal lexica…
Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification
Yi Zhu, Ehsan Shareghi, Yingzhen Li +2
Semi-supervised learning through deep generative models and multi-lingual pretraining techniques have orchestrated tremendous success across different areas of NLP. Nonetheless, th…