15 citations · 16 across the 14 of their papers we have counts for
13 papers · 1 filter
Embedding Models Measure in Peculiar Ways
Juri Opitz, Andrianos Michail
Embedding spaces define notions of semantic similarity and distance. We study whether those embeddings reflect physical measurements of mass, distance, time and volume, which admit…
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairs
Andrianos Michail, Stylianos Psychias, Michelle Wastl +3
Text embeddings are standard for semantic similarity tasks, yet their evaluation remains an open challenge. Current benchmarks are static, cover only a limited set of languages, ar…
ICDAR 2026 HIPE-OCRepair Competition on LLM-Assisted OCR Post-Correction for Historical Documents
Maud Ehrmann, Emanuela Boros, Juri Opitz +3
We present the results of HIPE-OCRepair-2026, an ICDAR competition on LLM-assisted OCR post-correction of historical documents. OCR post-correction remains a long-standing challeng…
Overview of HIPE-2026: Person-Place Relation Extraction from Multilingual Historical Texts
Juri Opitz, Maud Ehrmann, Corina Raclé +3
Was this person ever at that place, and if so, when? Answering such questions from noisy, multilingual historical documents is the central challenge of HIPE-2026, the third edition…
Information Representation Fairness in Long-Document Embeddings: The Peculiar Interaction of Positional and Language Bias
Elias Schuhmacher, Andrianos Michail, Juri Opitz +2
To be discoverable in an embedding-based search process, each part of a document should be reflected in its embedding representation. To quantify any potential reflection biases, w…
Robustness of Misinformation Classification Systems to Adversarial Examples Through BeamAttack
Arnisa Fazla, Lucas Krauter, David Guzman Piedrahita +1
We extend BeamAttack, an adversarial attack algorithm designed to evaluate the robustness of text classification systems through word-level modifications guided by beam search. Our…
Adapting Multilingual Embedding Models to Historical Luxembourgish
Andrianos Michail, Corina Julia Raclé, Juri Opitz +1
The growing volume of digitized historical texts requires effective semantic search using text embeddings. However, pre-trained multilingual models face challenges with historical…
Interpretable Text Embeddings and Text Similarity Explanation: A Survey
Juri Opitz, Lucas Möller, Andrianos Michail +2
Text embeddings are a fundamental component in many NLP tasks, including classification, regression, clustering, and semantic search. However, despite their ubiquitous application,…
Sentence Smith: Controllable Edits for Evaluating Text Embeddings
Hongji Li, Andrianos Michail, Reto Gubelmann +2
Controllable and transparent text generation has been a long-standing goal in NLP. Almost as long-standing is a general idea for addressing this challenge: Parsing text to a symbol…
MMTEB: Massive Multilingual Text Embedding Benchmark
Kenneth Enevoldsen, Isaac Chung, Imene Kerboua +83
Text embeddings are typically evaluated on a limited set of tasks, which are constrained by language, domain, and task diversity. To address these limitations and provide a more co…
Examining Multilingual Embedding Models Cross-Lingually Through LLM-Generated Adversarial Examples
Andrianos Michail, Simon Clematide, Rico Sennrich
The evaluation of cross-lingual semantic search models is often limited to existing datasets from tasks such as information retrieval and semantic textual similarity. We introduce…
PARAPHRASUS : A Comprehensive Benchmark for Evaluating Paraphrase Detection Models
Andrianos Michail, Simon Clematide, Juri Opitz
The task of determining whether two texts are paraphrases has long been a challenge in NLP. However, the prevailing notion of paraphrase is often quite simplistic, offering only a…