268 citations · 662 across the 69 of their papers we have counts for
15 papers · 2 filters
On Cross-Lingual Retrieval with Multilingual Text Encoders
Robert Litschko, Ivan Vulić, Simone Paolo Ponzetto +1
In this work we present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval task…
Prix-LM: Pretraining for Multilingual Knowledge Base Construction
Wenxuan Zhou, Fangyu Liu, Ivan Vulić +2
Knowledge bases (KBs) contain plenty of structured world and commonsense knowledge. As such, they often complement distributional text-based information and facilitate various down…
Composable Sparse Fine-Tuning for Cross-Lingual Transfer
Alan Ansell, Edoardo Maria Ponti, Anna Korhonen +1
Fine-tuning the entire set of parameters of a large pretrained model has become the mainstream approach for transfer learning. To increase its efficiency and prevent catastrophic f…
ConvFiT: Conversational Fine-Tuning of Pretrained Language Models
Ivan Vulić, Pei-Hao Su, Sam Coope +5
Transformer-based language models (LMs) pretrained on large text collections are proven to store a wealth of semantic knowledge. However, 1) they are not effective as sentence enco…
MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models
Qianchu Liu, Fangyu Liu, Nigel Collier +2
Recent work indicated that pretrained language models (PLMs) such as BERT and RoBERTa can be transformed into effective sentence and word encoders even via simple self-supervised t…
xGQA: Cross-Lingual Visual Question Answering
Jonas Pfeiffer, Gregor Geigle, Aishwarya Kamath +4
Recent advances in multimodal vision and language modeling have predominantly focused on the English language, mostly due to the lack of multilingual multimodal datasets to steer m…
Towards Zero-shot Language Modeling
Edoardo Maria Ponti, Ivan Vulić, Ryan Cotterell +2
Can we construct a neural model that is inductively biased towards learning human languages? Motivated by this question, we aim at constructing an informative prior over neural wei…
Modelling Latent Translations for Cross-Lingual Transfer
Edoardo Maria Ponti, Julia Kreutzer, Ivan Vulić +1
While achieving state-of-the-art results in multiple tasks and languages, translation-based cross-lingual transfer is often overlooked in favour of massively multilingual pre-train…
RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Models
Soumya Barikeri, Anne Lauscher, Ivan Vulić +1
Text representation models are prone to exhibit a range of societal biases, reflecting the non-controlled and biased nature of the underlying pretraining data, which consequently l…
Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking
Fangyu Liu, Ivan Vulić, Anna Korhonen +1
Injecting external domain-specific knowledge (e.g., UMLS) into pretrained language models (LMs) advances their capability to handle specialised in-domain tasks such as biomedical e…
Multilingual and Cross-Lingual Intent Detection from Spoken Data
Daniela Gerz, Pei-Hao Su, Razvan Kusztos +6
We present a systematic study on multilingual and cross-lingual intent detection from spoken data. The study leverages a new resource put forth in this work, termed MInDS-14, a fir…
AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples
Qianchu Liu, Edoardo M. Ponti, Diana McCarthy +2
Capturing word meaning in context and distinguishing between correspondences and variations across languages is key to building successful multilingual and cross-lingual text repre…
Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems
Evgeniia Razumovskaia, Goran Glavaš, Olga Majewska +3
In task-oriented dialogue (ToD), a user holds a conversation with an artificial agent to complete a concrete task. Although this technology represents one of the central objectives…
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders
Fangyu Liu, Ivan Vulić, Anna Korhonen +1
Pretrained Masked Language Models (MLMs) have revolutionised NLP in recent years. However, previous work has indicated that off-the-shelf MLMs are not effective as universal lexica…
Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval
Robert Litschko, Ivan Vulić, Simone Paolo Ponzetto +1
Pretrained multilingual text encoders based on neural Transformer architectures, such as multilingual BERT (mBERT) and XLM, have achieved strong performance on a myriad of language…