19 citations · 34 across the 22 of their papers we have counts for
18 papers · 1 filter
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
VetScore: Risk-Weighted Fact Verification for Veterinary Long-Form QA with Citations
Ivan Kartáč, Jan Tovarys, Mateusz Lango +1
Citation excerpts can be used to increase the reliability of generated outputs and their faithfulness to cited sources, which is especially important in high-stakes domains such as…
PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages
Daryna Dementieva, Nikolay Babakov, Kathy Hämmerl +14
Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models (LLMs), yet existing benchmarks remain heavily biased toward high-resourc…
UFAL-CUNI at SemEval-2026 Task 11: An Efficient Modular Neuro-symbolic Method for Syllogistic Reasoning
Ivan Kartáč, Kristýna Onderková, Jan Bronec +3
This paper describes our system submitted to SemEval-2026 Task 11: Disentangling Content and Formal Reasoning in Large Language Models. We present an efficient modular neuro-symbol…
Reasoning Gets Harder for LLMs Inside A Dialogue
Ivan Kartáč, Mateusz Lango, Ondřej Dušek
Large Language Models (LLMs) achieve strong performance on many reasoning benchmarks, yet these evaluations typically focus on isolated tasks that differ from real-world usage in t…
SRS-Stories: Vocabulary-constrained multilingual story generation for language learning
Wiktor Kamzela, Mateusz Lango, Ondrej Dusek
In this paper, we use large language models to generate personalized stories for language learners, using only the vocabulary they know. The generated texts are specifically writte…
LLM Agents Implement an NLG System from Scratch: Building Interpretable Rule-Based RDF-to-Text Generators
Mateusz Lango, Ondřej Dušek
We present a novel neurosymbolic framework for RDF-to-text generation, in which the model is "trained" through collaborative interactions among multiple LLM agents rather than trad…
OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMs
Ivan Kartáč, Mateusz Lango, Ondřej Dušek
Large Language Models (LLMs) have demonstrated great potential as evaluators of NLG systems, allowing for high-quality, reference-free, and multi-aspect assessments. However, exist…
Leveraging Large Language Models for Building Interpretable Rule-Based Data-to-Text Systems
Jędrzej Warczyński, Mateusz Lango, Ondrej Dusek
We introduce a simple approach that uses a large language model (LLM) to automatically implement a fully interpretable rule-based data-to-text system in pure Python. Experimental e…
Polish-ASTE: Aspect-Sentiment Triplet Extraction Datasets for Polish
Marta Lango, Borys Naglik, Mateusz Lango +1
Aspect-Sentiment Triplet Extraction (ASTE) is one of the most challenging and complex tasks in sentiment analysis. It concerns the construction of triplets that contain an aspect,…
ASTE Transformer Modelling Dependencies in Aspect-Sentiment Triplet Extraction
Iwo Naglik, Mateusz Lango
Aspect-Sentiment Triplet Extraction (ASTE) is a recently proposed task of aspect-based sentiment analysis that consists in extracting (aspect phrase, opinion phrase, sentiment pola…
A Survey of Text Style Transfer: Applications and Ethical Implications
Sourabrata Mukherjee, Mateusz Lango, Zdenek Kasner +1
Text style transfer (TST) is an important task in controllable text generation, which aims to control selected attributes of language use, such as politeness, formality, or sentime…
Generating clickbait spoilers with an ensemble of large language models
Mateusz Woźny, Mateusz Lango
Clickbait posts are a widespread problem in the webspace. The generation of spoilers, i.e. short texts that neutralize clickbait by providing information that satisfies the curiosi…
Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs
Simone Balloccu, Patrícia Schmidtová, Mateusz Lango +1
Natural Language Processing (NLP) research is increasingly focusing on the use of Large Language Models (LLMs), with some of the most popular ones being either fully or partially c…
The Problem of Coherence in Natural Language Explanations of Recommendations
Jakub Raczyński, Mateusz Lango, Jerzy Stefanowski
Providing natural language explanations for recommendations is particularly useful from the perspective of a non-expert user. Although several methods for providing such explanatio…
Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generation
Mateusz Lango, Ondřej Dušek
Hallucination of text ungrounded in the input is a well-known problem in neural data-to-text generation. Many methods have been proposed to mitigate it, but they typically require…
With a Little Help from the Authors: Reproducing Human Evaluation of an MT Error Detector
Ondřej Plátek, Mateusz Lango, Ondřej Dušek
This work presents our efforts to reproduce the results of the human evaluation experiment presented in the paper of Vamvas and Sennrich (2022), which evaluated an automatic system…
Three Ways of Using Large Language Models to Evaluate Chat
Ondřej Plátek, Vojtěch Hudeček, Patricia Schmidtová +2
This paper describes the systems submitted by team6 for ChatEval, the DSTC 11 Track 4 competition. We present three different approaches to predicting turn-level qualities of chatb…