3 citations · 3 across the 9 of their papers we have counts for
10 papers
Multilingual GSM-Symbolic: What determines capability transfer across languages?
Kenneth Enevoldsen, Riley Herchert, Sofie Mosegaard +22
We understand little about how capabilities acquired in one language carry over to another, or what governs this transfer: evaluations rely on incomparable, saturation-prone datase…
Belief Propagation in LLM World Models: Measuring Strategic Information Bias with Prediction Markets
Mykola Khandoga, Yevhen Kostiuk, Anton Polishko +4
Every information ecosystem produces beliefs that shape strategic decisions. Both human analysts and AI systems inherit the blind spots of their information sources. We show that L…
Naturalistic measure of social norms alignment
Yevhen Kostiuk, Kenneth Enevoldsen, Peter Bjerregaard Vahlstrup +2
Social norms reflect shared expectations on acceptable behavior. Measuring social norms alignment remains challenging, with existing approaches typically relying on artificial clos…
One prompt is not enough: Instruction Sensitivity Undermines Embedding Model Evaluation
Yevhen Kostiuk, Kenneth Enevoldsen
Instruction embedding models have become common among state-of-the-art models, however are evaluated using a single prompt per task. The single-point evaluation ignores a main prob…
The Veln(ia)s is in the Details: Evaluating LLM Judgment on Latvian and Lithuanian Short Answer Matching
Yevhen Kostiuk, Oxana Vitman, Łukasz Gagała +1
In this work, we address the challenge of evaluating large language models (LLMs) on the short answer matching task for Latvian and Lithuanian languages. We introduce novel dataset…
Towards Multilingual LLM Evaluation for Baltic and Nordic languages: A study on Lithuanian History
Yevhen Kostiuk, Oxana Vitman, Łukasz Gagała +1
In this work, we evaluated Lithuanian and general history knowledge of multilingual Large Language Models (LLMs) on a multiple-choice question-answering task. The models were teste…
From English-Centric to Effective Bilingual: LLMs with Custom Tokenizers for Underrepresented Languages
Artur Kiulian, Anton Polishko, Mykola Khandoga +10
In this paper, we propose a model-agnostic cost-effective approach to developing bilingual base large language models (LLMs) to support English and any target language. The method…
SpaDeLeF: A Dataset for Hierarchical Classification of Lexical Functions for Collocations in Spanish
Yevhen Kostiuk, Grigori Sidorov, Olga Kolesnikova
In natural language processing (NLP), lexical function is a concept to unambiguously represent semantic and syntactic features of words and phrases in text first crafted in the Mea…
Automatic Translation of Hate Speech to Non-hate Speech in Social Media Texts
Yevhen Kostiuk, Atnafu Lambebo Tonja, Grigori Sidorov +1
In this paper, we investigate the issue of hate speech by presenting a novel task of translating hate speech into non-hate speech text while preserving its meaning. As a case study…
Sarcasm Detection Framework Using Context, Emotion and Sentiment Features
Oxana Vitman, Yevhen Kostiuk, Grigori Sidorov +1
Sarcasm detection is an essential task that can help identify the actual sentiment in user-generated data, such as discussion forums or tweets. Sarcasm is a sophisticated form of l…