70 citations · 97 across the 15 of their papers we have counts for
16 papers
CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA
Hanseok Oh, Parishad BehnamGhader, Benno Krojer +4
Knowledge-Intensive Visual Question Answering (KI-VQA) benchmarks evaluate Vision-Language Models (VLMs) as multimodal knowledge assistants by requiring external information beyond…
Leveraging Routing Dynamics in Mixture-of-Experts Models for Efficient Language Adaptation
Aditi Khandelwal, Marius Mosbach, Verna Dankers +2
Mixture-of-Experts (MoE) models are widely used to scale language models, yet their expert routing behavior and adaptation in a multilingual setting remain underexplored. In this w…
Memorization Inheritance in Sequence-Level Knowledge Distillation for Neural Machine Translation
Verna Dankers, Vikas Raunak
In this work, we explore how instance-level memorization in the teacher Neural Machine Translation (NMT) model gets inherited by the student model in sequence-level knowledge disti…
Generalisation First, Memorisation Second? Memorisation Localisation for Natural Language Classification Tasks
Verna Dankers, Ivan Titov
Memorisation is a natural part of learning from real-world data: neural models pick up on atypical input-output combinations and store those training examples in their parameter sp…
Evaluating Subword Tokenization: Alien Subword Composition and OOV Generalization Challenge
Khuyagbaatar Batsuren, Ekaterina Vylomova, Verna Dankers +4
The popular subword tokenizers of current language models, such as Byte-Pair Encoding (BPE), are known not to respect morpheme boundaries, which affects the downstream performance…
Latent Feature-based Data Splits to Improve Generalisation Evaluation: A Hate Speech Detection Case Study
Maike Züfle, Verna Dankers, Ivan Titov
With the ever-growing presence of social media platforms comes the increased spread of harmful content and the need for robust hate speech detection systems. Such systems easily ov…
Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation
Verna Dankers, Ivan Titov, Dieuwke Hupkes
When training a neural network, it will quickly memorise some source-target mappings from your dataset but never learn some others. Yet, memorisation is not easily expressed as a b…
Non-Compositionality in Sentiment: New Data and Analyses
Verna Dankers, Christopher G. Lucas
When natural language phrases are combined, their meaning is often more than the sum of their parts. In the context of NLP tasks such as sentiment analysis, where the meaning of a…
Recursive Neural Networks with Bottlenecks Diagnose (Non-)Compositionality
Verna Dankers, Ivan Titov
A recent line of work in NLP focuses on the (dis)ability of models to generalise compositionally for artificial languages. However, when considering natural language tasks, the dat…
Text Characterization Toolkit
Daniel Simig, Tianlu Wang, Verna Dankers +4
In NLP, models are usually evaluated by reporting single-number performance scores on a number of readily available benchmarks, without much deeper analysis. Here, we argue that -…
State-of-the-art generalisation research in NLP: A taxonomy and review
Dieuwke Hupkes, Mario Giulianelli, Verna Dankers +17
The ability to generalise well is one of the primary desiderata of natural language processing (NLP). Yet, what 'good generalisation' entails and how it should be evaluated is not…
Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation
Verna Dankers, Christopher G. Lucas, Ivan Titov
Unlike literal expressions, idioms' meanings do not directly follow from their parts, posing a challenge for neural machine translation (NMT). NMT models are often unable to transl…
The paradox of the compositionality of natural language: a neural machine translation case study
Verna Dankers, Elia Bruni, Dieuwke Hupkes
Obtaining human-like performance in NLP is often argued to require compositional generalisation. Whether neural networks exhibit this ability is usually studied by training models…
Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsing
Anna Langedijk, Verna Dankers, Phillip Lippe +4
Meta-learning, or learning to learn, is a technique that can help to overcome resource scarcity in cross-lingual NLP problems, by enabling fast adaptation to new tasks. We apply mo…
Compositionality decomposed: how do neural networks generalise?
Dieuwke Hupkes, Verna Dankers, Mathijs Mul +1
Despite a multitude of empirical studies, little consensus exists on whether neural networks are able to generalise compositionally, a controversy that, in part, stems from a lack…
Transcoding compositionally: using attention to find more generalizable solutions
Kris Korrel, Dieuwke Hupkes, Verna Dankers +1
While sequence-to-sequence models have shown remarkable generalization power across several natural language tasks, their construct of solutions are argued to be less compositional…