79 citations · 401 across the 59 of their papers we have counts for
66 papers
WiC is Not WSD: A Study on LLMs and Lexical Ambiguity Resolution
Yi Zhou, Kiamehr Rezaee, Danushka Bollegala +2
Word-in-Context (WiC) remains challenging for language models, despite recent progress on lexical-semantic tasks. We hypothesise that this difficulty arises not only from comparing…
Exploring State Tracking Capabilities of Large Language Models
Kiamehr Rezaee, Jose Camacho-Collados, Mohammad Taher Pilehvar
Large Language Models (LLMs) have demonstrated impressive capabilities in solving complex tasks, including those requiring a certain level of reasoning. In this paper, we focus on…
Pun Unintended: LLMs and the Illusion of Humor Understanding
Alessandro Zangari, Matteo Marcuzzo, Andrea Albarelli +2
Puns are a form of humorous wordplay that exploits polysemy and phonetic similarity. While LLMs have shown promise in detecting puns, we show in this paper that their understanding…
MORABLES: A Benchmark for Assessing Abstract Moral Reasoning in LLMs with Fables
Matteo Marcuzzo, Alessandro Zangari, Andrea Albarelli +2
As LLMs excel on standard reading comprehension benchmarks, attention is shifting toward evaluating their capacity for complex abstract reasoning and inference. Literature-based be…
Automatic Extraction of Metaphoric Analogies from Literary Texts: Task Formulation, Dataset Construction, and Evaluation
Joanne Boisson, Zara Siddique, Hsuvas Borkakoty +3
Extracting metaphors and analogies from free text requires high-level reasoning abilities such as abstraction and language understanding. Our study focuses on the extraction of the…
Sensitive Content Classification in Social Media: A Holistic Resource and Evaluation
Dimosthenis Antypas, Indira Sen, Carla Perez-Almendros +2
The detection of sensitive content in large datasets is crucial for ensuring that shared and analysed data is free from harmful material. However, current moderation tools, such as…
MetaphorShare: A Dynamic Collaborative Repository of Open Metaphor Datasets
Joanne Boisson, Arif Mehmood, Jose Camacho-Collados
The metaphor studies community has developed numerous valuable labelled corpora in various languages over the years. Many of these resources are not only unknown to the NLP communi…
Multilingual Topic Classification in X: Dataset and Analysis
Dimosthenis Antypas, Asahi Ushio, Francesco Barbieri +1
In the dynamic realm of social media, diverse topics are discussed daily, transcending linguistic boundaries. However, the complexities of understanding and categorising this conte…
Analysing Zero-Shot Readability-Controlled Sentence Simplification
Abdullah Barayan, Jose Camacho-Collados, Fernando Alva-Manchego
Readability-controlled text simplification (RCTS) rewrites texts to lower readability levels while preserving their meaning. RCTS models often depend on parallel corpora with reada…
Evaluating Short-Term Temporal Fluctuations of Social Biases in Social Media Data and Masked Language Models
Yi Zhou, Danushka Bollegala, Jose Camacho-Collados
Social biases such as gender or racial biases have been reported in language models (LMs), including Masked Language Models (MLMs). Given that MLMs are continuously trained with in…
BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages
Junho Myung, Nayeon Lee, Yi Zhou +19
Large language models (LLMs) often lack culture-specific knowledge of daily life, especially across diverse regions and non-English languages. Existing benchmarks for evaluating LL…
A Systematic Analysis on the Temporal Generalization of Language Models in Social Media
Asahi Ushio, Jose Camacho-Collados
In machine learning, temporal shifts occur when there are differences between training and test splits in terms of time. For streaming data such as news or social media, models are…
Causal Effects of Trigger Words in Social Media Discussions: A Large-Scale Case Study about UK Politics on Reddit
Dimosthenis Antypas, Christian Arnold, Nedjma Ousidhoum +2
Political debates on social media often escalate quickly, leading to increased engagement as well as more emotional and polarised exchanges. Trigger points (Mau, Lux, and Westheuse…
Language Models for Text Classification: Is In-Context Learning Enough?
Aleksandra Edwards, Jose Camacho-Collados
Recent foundational language models have shown state-of-the-art performance in many NLP tasks in zero- and few-shot settings. An advantage of these models over more standard approa…
DANSK and DaCy 2.6.0: Domain Generalization of Danish Named Entity Recognition
Kenneth Enevoldsen, Emil Trenckner Jessen, Rebekah Baglini
Named entity recognition is one of the cornerstones of Danish NLP, essential for language technology applications within both industry and research. However, Danish NER is inhibite…
Construction Artifacts in Metaphor Identification Datasets
Joanne Boisson, Luis Espinosa-Anke, Jose Camacho-Collados
Metaphor identification aims at understanding whether a given expression is used figuratively in context. However, in this paper we show how existing metaphor identification datase…
SuperTweetEval: A Challenging, Unified and Heterogeneous Benchmark for Social Media NLP Research
Dimosthenis Antypas, Asahi Ushio, Francesco Barbieri +5
Despite its relevance, the maturity of NLP for social media pales in comparison with general-purpose models, metrics and benchmarks. This fragmented landscape makes it hard for the…
A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models
Yi Zhou, Jose Camacho-Collados, Danushka Bollegala
Various types of social biases have been reported with pretrained Masked Language Models (MLMs) in prior work. However, multiple underlying factors are associated with an MLM such…
RelBERT: Embedding Relations with Language Models
Asahi Ushio, Jose Camacho-Collados, Steven Schockaert
Many applications need access to background knowledge about how different concepts and entities are related. Although Knowledge Graphs (KG) and Large Language Models (LLM) can addr…
Tweet Insights: A Visualization Platform to Extract Temporal Insights from Twitter
Daniel Loureiro, Kiamehr Rezaee, Talayeh Riahi +4
This paper introduces a large collection of time series data derived from Twitter, postprocessed using word embedding techniques, as well as specialized fine-tuned language models.…
Robust Hate Speech Detection in Social Media: A Cross-Dataset Empirical Evaluation
Dimosthenis Antypas, Jose Camacho-Collados
The automatic detection of hate speech online is an active research area in NLP. Most of the studies to date are based on social media datasets that contribute to the creation of h…
A Practical Toolkit for Multilingual Question and Answer Generation
Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados
Generating questions along with associated answers from a text has applications in several domains, such as creating reading comprehension tests for students, or improving document…
An Empirical Comparison of LM-based Question and Answer Generation Methods
Asahi Ushio, Fernando Alva-Manchego, Jose Camacho-Collados
Question and answer generation (QAG) consists of generating a set of question-answer pairs given a context (e.g. a paragraph). This task has a variety of applications, such as data…
An Efficient Multilingual Language Model Compression through Vocabulary Trimming
Asahi Ushio, Yi Zhou, Jose Camacho-Collados
Multilingual language model (LM) have become a powerful tool in NLP especially for non-English languages. Nevertheless, model parameters of multilingual LMs remain large due to the…