565 citations · 834 across the 31 of their papers we have counts for
35 papers
What do Reward Models Memorize?
Ivo Verhoeven, Pushkar Mishra, Ekaterina Shutova
This paper studies what discriminatively trained reward models (RMs) memorize by measuring counterfactual memorization on two human preference datasets. We show that RMs 1) misallo…
Yesterday's News: Benchmarking Multi-Dimensional Out-of-Distribution Generalization of Misinformation Detection Models
Ivo Verhoeven, Pushkar Mishra, Ekaterina Shutova
This article introduces misinfo-general, a benchmark dataset for evaluating misinformation models' ability to perform out-of-distribution generalization. Misinformation changes rap…
A (More) Realistic Evaluation Setup for Generalisation of Community Models on Malicious Content Detection
Ivo Verhoeven, Pushkar Mishra, Rahel Beloch +2
Community models for malicious content detection, which take into account the context from a social graph alongside the content itself, have shown remarkable performance on benchma…
What's the Meaning of Superhuman Performance in Today's NLU?
Simone Tedeschi, Johan Bos, Thierry Declerck +9
In the last five years, there has been a significant focus in Natural Language Processing (NLP) on developing larger Pretrained Language Models (PLMs) and introducing benchmarks su…
How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning
Rochelle Choenni, Dan Garrette, Ekaterina Shutova
Multilingual large language models (MLLMs) are jointly trained on data from many different languages such that representation of individual languages can benefit from other languag…
CK-Transformer: Commonsense Knowledge Enhanced Transformers for Referring Expression Comprehension
Zhi Zhang, Helen Yannakoudakis, Xiantong Zhen +1
The task of multimodal referring expression comprehension (REC), aiming at localizing an image region described by a natural language expression, has recently received increasing a…
FewShotTextGCN: K-hop neighborhood regularization for few-shot learning on graphs
Niels van der Heijden, Ekaterina Shutova, Helen Yannakoudakis
We present FewShotTextGCN, a novel method designed to effectively utilize the properties of word-document graphs for improved learning in low-resource settings. We introduce K-hop…
Scientific and Creative Analogies in Pretrained Language Models
Tamara Czinczoll, Helen Yannakoudakis, Pushkar Mishra +1
This paper examines the encoding of analogy in large-scale pretrained language models, such as BERT and GPT-2. Existing analogy datasets typically focus on a limited set of analogi…
Data-Efficient Cross-Lingual Transfer with Language-Specific Subnetworks
Rochelle Choenni, Dan Garrette, Ekaterina Shutova
Large multilingual language models typically share their parameters across all languages, which enables cross-lingual task transfer, but learning can also be hindered when training…
Learning New Tasks from a Few Examples with Soft-Label Prototypes
Avyav Kumar Singh, Ekaterina Shutova, Helen Yannakoudakis
Existing approaches to few-shot learning in NLP rely on large language models (LLMs) and/or fine-tuning of these to generalise on out-of-distribution data. In this work, we propose…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you?
Rochelle Choenni, Ekaterina Shutova, Robert van Rooij
In this paper, we investigate what types of stereotypical information are captured by pretrained language models. We present the first dataset comprising stereotypical attributes o…
Meta-Learning with Variational Semantic Memory for Word Sense Disambiguation
Yingjun Du, Nithin Holla, Xiantong Zhen +2
A critical challenge faced by supervised word sense disambiguation (WSD) is the lack of large annotated datasets with sufficient coverage of words in their diversity of senses. Thi…
Ruddit: Norms of Offensiveness for English Reddit Comments
Rishav Hada, Sohi Sudhir, Pushkar Mishra +3
On social media platforms, hateful and offensive language negatively impact the mental well-being of users and the participation of people from diverse backgrounds. Automatic metho…
Modeling Users and Online Communities for Abuse Detection: A Position on Ethics and Explainability
Pushkar Mishra, Helen Yannakoudakis, Ekaterina Shutova
Abuse on the Internet is an important societal problem of our time. Millions of Internet users face harassment, racism, personal attacks, and other types of abuse across various pl…
How Metaphors Impact Political Discourse: A Large-Scale Topic-Agnostic Study Using Neural Metaphor Detection
Vinodkumar Prabhakaran, Marek Rei, Ekaterina Shutova
Metaphors are widely used in political rhetoric as an effective framing device. While the efficacy of specific metaphors such as the war metaphor in political discourse has been do…
Meta-Learning for Fast Cross-Lingual Adaptation in Dependency Parsing
Anna Langedijk, Verna Dankers, Phillip Lippe +4
Meta-learning, or learning to learn, is a technique that can help to overcome resource scarcity in cross-lingual NLP problems, by enabling fast adaptation to new tasks. We apply mo…
Us vs. Them: A Dataset of Populist Attitudes, News Bias and Emotions
Pere-Lluís Huguet-Cabot, David Abadi, Agneta Fischer +1
Computational modelling of political discourse tasks has become an increasingly important area of research in natural language processing. Populist rhetoric has risen across the po…
Multilingual and cross-lingual document classification: A meta-learning approach
Niels van der Heijden, Helen Yannakoudakis, Pushkar Mishra +1
The great majority of languages in the world are considered under-resourced for the successful application of deep learning methods. In this work, we propose a meta-learning approa…
A Multimodal Framework for the Detection of Hateful Memes
Phillip Lippe, Nithin Holla, Shantanu Chandra +4
An increasingly common expression of online hate speech is multimodal in nature and comes in the form of memes. Designing systems to automatically detect hateful content is of para…
Cross-neutralising: Probing for joint encoding of linguistic information in multilingual models
Rochelle Choenni, Ekaterina Shutova
Multilingual sentence encoders are widely used to transfer NLP models across languages. The success of this transfer is, however, dependent on the model's ability to encode the pat…
What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties
Rochelle Choenni, Ekaterina Shutova
Multilingual sentence encoders have seen much success in cross-lingual model transfer for downstream NLP tasks. Yet, we know relatively little about the properties of individual la…
Meta-Learning with Sparse Experience Replay for Lifelong Language Learning
Nithin Holla, Pushkar Mishra, Helen Yannakoudakis +1
Lifelong learning requires models that can continuously learn from sequential streams of data without suffering catastrophic forgetting due to shifts in data distributions. Deep le…
Graph-based Modeling of Online Communities for Fake News Detection
Shantanu Chandra, Pushkar Mishra, Helen Yannakoudakis +3
Over the past few years, there has been a substantial effort towards automated detection of fake news on social media platforms. Existing research has modeled the structure, style,…