15 citations · 19 across the 9 of their papers we have counts for
9 papers
MMTEB: Massive Multilingual Text Embedding Benchmark
Kenneth Enevoldsen, Isaac Chung, Imene Kerboua +83
Text embeddings are typically evaluated on a limited set of tasks, which are constrained by language, domain, and task diversity. To address these limitations and provide a more co…
LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Inconsistencies
Felix Friedrich, Simone Tedeschi, Patrick Schramowski +5
Building safe Large Language Models (LLMs) across multiple languages is essential in ensuring both safe access and linguistic diversity. To this end, we conduct a large-scale, comp…
Truth or Mirage? Towards End-to-End Factuality Evaluation with LLM-Oasis
Alessandro Scirè, Andrei Stefan Bejgu, Simone Tedeschi +3
After the introduction of Large Language Models (LLMs), there have been substantial improvements in the performance of Natural Language Generation (NLG) tasks, including Text Summa…
ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming
Simone Tedeschi, Felix Friedrich, Patrick Schramowski +4
When building Large Language Models (LLMs), it is paramount to bear safety in mind and protect them with guardrails. Indeed, LLMs should never generate content promoting or normali…
Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code
Taishi Nakamura, Mayank Mishra, Simone Tedeschi +42
Pretrained language models are an integral part of AI applications, but their high computational cost for training limits accessibility. Initiatives such as Bloom and StarCoder aim…
RED: a Filtered and Multilingual Relation Extraction Dataset
Pere-Lluís Huguet Cabot, Simone Tedeschi, Axel-Cyrille Ngonga Ngomo +1
Relation Extraction (RE) is a task that identifies relationships between entities in a text, enabling the acquisition of relational facts and bridging the gap between natural langu…
What's the Meaning of Superhuman Performance in Today's NLU?
Simone Tedeschi, Johan Bos, Thierry Declerck +9
In the last five years, there has been a significant focus in Natural Language Processing (NLP) on developing larger Pretrained Language Models (PLMs) and introducing benchmarks su…
EUREKA: EUphemism Recognition Enhanced through Knn-based methods and Augmentation
Sedrick Scott Keh, Rohit K. Bharadwaj, Emmy Liu +3
We introduce EUREKA, an ensemble-based approach for performing automatic euphemism detection. We (1) identify and correct potentially mislabelled rows in the dataset, (2) curate an…
Focusing on Context is NICE: Improving Overshadowed Entity Disambiguation
Vera Provatorova, Simone Tedeschi, Svitlana Vakulenko +2
Entity disambiguation (ED) is the task of mapping an ambiguous entity mention to the corresponding entry in a structured knowledge base. Previous research showed that entity oversh…