144 citations · 209 across the 23 of their papers we have counts for
25 papers
Pre-AF 13: An Interpretable Atrial Fibrillation Risk Score Mined from Discharge Reports
Olga Shakhmatova, Dmitrii Kriukov, Daniil Larionov +10
Background. Atrial fibrillation (AF) is the most prevalent cardiac arrhythmia and a major determinant of prognosis. Established AF risk scores rely on factors (older age, hypertens…
A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs
Artem Shelmanov, Ekaterina Fadeeva, Akim Tsvigun +9
Large Language Models (LLMs) have the tendency to hallucinate, i.e., to sporadically generate false or fabricated information. This presents a major challenge, as hallucinations of…
Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Heads
Artem Vazhentsev, Lyudmila Rvanova, Gleb Kuzmin +8
While large language models (LLMs) have become highly capable, they remain prone to factual inaccuracies, commonly referred to as "hallucinations." Uncertainty quantification (UQ)…
GenAI Content Detection Task 1: English and Multilingual Machine-Generated Text Detection: AI vs. Human
Yuxia Wang, Artem Shelmanov, Jonibek Mansurov +23
We present the GenAI Content Detection Task~1 -- a shared task on binary machine generated text detection, conducted as a part of the GenAI workshop at COLING 2025. The task consis…
Argument-Based Comparative Question Answering Evaluation Benchmark
Irina Nikishina, Saba Anwar, Nikolay Dolgov +6
In this paper, we aim to solve the problems standing in the way of automatic comparative question answering. To this end, we propose an evaluation framework to assess the quality o…
Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AI
Yuxia Wang, Rui Xing, Jonibek Mansurov +23
Prior studies have shown that distinguishing text generated by Large Language Models (LLMs) from human-written one is highly challenging for humans, and often no better than random…
Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistency
Roman Vashurin, Maiya Goloburda, Albina Ilina +4
Uncertainty quantification (UQ) methods for Large Language Models (LLMs) encompass a variety of approaches, with two major types being particularly prominent: information-based, wh…
Unconditional Truthfulness: Learning Unconditional Uncertainty of Large Language Models
Artem Vazhentsev, Ekaterina Fadeeva, Rui Xing +7
Uncertainty quantification (UQ) has emerged as a promising approach for detecting hallucinations and low-quality output of Large Language Models (LLMs). However, obtaining proper u…
LLM-DetectAIve: a Tool for Fine-Grained Machine-Generated Text Detection
Mervat Abassy, Kareem Elozeiri, Alexander Aziz +21
The ease of access to large language models (LLMs) has enabled a widespread of machine-generated texts, and now it is often hard to tell whether a piece of text was human-written o…
Inference-Time Selective Debiasing to Enhance Fairness in Text Classification Models
Gleb Kuzmin, Neemesh Yadav, Ivan Smirnov +2
We propose selective debiasing -- an inference-time safety mechanism designed to enhance the overall model quality in terms of prediction performance and fairness, especially in sc…
Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph
Roman Vashurin, Ekaterina Fadeeva, Artem Vazhentsev +12
The rapid proliferation of large language models (LLMs) has stimulated researchers to seek effective and efficient approaches to deal with LLM hallucinations and low-quality output…
Vikhr: The Family of Open-Source Instruction-Tuned Large Language Models for Russian
Aleksandr Nikolich, Konstantin Korolev, Sergei Bratchikov +2
There has been a surge in the development of various Large Language Models (LLMs). However, text generation for languages other than English often faces significant challenges, inc…
SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection
Yuxia Wang, Jonibek Mansurov, Petar Ivanov +12
We present the results and the main findings of SemEval-2024 Task 8: Multigenerator, Multidomain, and Multilingual Machine-Generated Text Detection. The task featured three subtask…
Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification
Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov +9
Large language models (LLMs) are notorious for hallucinating, i.e., producing erroneous claims in their output. Such hallucinations can be dangerous, as occasional factual inaccura…
M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection
Yuxia Wang, Jonibek Mansurov, Petar Ivanov +11
The advent of Large Language Models (LLMs) has brought an unprecedented surge in machine-generated text (MGT) across diverse channels. This raises legitimate concerns about its pot…
LM-Polygraph: Uncertainty Estimation for Language Models
Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun +9
Recent advancements in the capabilities of large language models (LLMs) have paved the way for a myriad of groundbreaking applications in various fields. However, a significant cha…
M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection
Yuxia Wang, Jonibek Mansurov, Petar Ivanov +12
Large language models (LLMs) have demonstrated remarkable capability to generate fluent responses to a wide variety of user queries. However, this has also raised concerns about th…
Active Learning for Abstractive Text Summarization
Akim Tsvigun, Ivan Lysenko, Danila Sedashov +9
Construction of human-curated annotated datasets for abstractive text summarization (ATS) is very time-consuming and expensive because creating each instance requires a human annot…
Medical Image Captioning via Generative Pretrained Transformers
Alexander Selivanov, Oleg Y. Rogov, Daniil Chesakov +3
The automatic clinical caption generation problem is referred to as proposed model combining the analysis of frontal chest X-Ray scans with structured patient information from the…
NeuralSympCheck: A Symptom Checking and Disease Diagnostic Neural Model with Logic Regularization
Aleksandr Nesterov, Bulat Ibragimov, Dmitriy Umerenkov +3
The symptom checking systems inquire users for their symptoms and perform a rapid and affordable medical assessment of their condition. The basic symptom checking systems based on…
Towards Computationally Feasible Deep Active Learning
Akim Tsvigun, Artem Shelmanov, Gleb Kuzmin +5
Active learning (AL) is a prominent technique for reducing the annotation effort required for training machine learning models. Deep learning offers a solution for several essentia…
Nonparametric Uncertainty Quantification for Single Deterministic Neural Network
Nikita Kotelevskii, Aleksandr Artemenkov, Kirill Fedyanin +6
This paper proposes a fast and scalable method for uncertainty quantification of machine learning models' predictions. First, we show the principled way to measure the uncertainty…
Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimates
Artem Shelmanov, Dmitri Puzyrev, Lyubov Kupriyanova +7
Annotating training data for sequence tagging of texts is usually very time-consuming. Recent advances in transfer learning for natural language processing in conjunction with acti…
Neural Entity Linking: A Survey of Models Based on Deep Learning
Ozge Sevgili, Artem Shelmanov, Mikhail Arkhipov +2
This survey presents a comprehensive description of recent neural entity linking (EL) systems developed since 2015 as a result of the "deep learning revolution" in natural language…