8 citations · 32 across the 18 of their papers we have counts for
18 papers
On Synthesizing Data for Context Attribution in Question Answering
Gorjan Radevski, Kiril Gashteovski, Shahbaz Syed +11
Question Answering (QA) accounts for a significant portion of LLM usage "in the wild". However, LLMs sometimes produce false or misleading responses, also known as "hallucinations"…
Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions
Emmy Liu, Amanda Bertsch, Lintang Sutawika +9
Improvements in language model capabilities are often attributed to increasing model size or training data, but in some cases smaller models trained on curated data or with differe…
MEDDxAgent: A Unified Modular Agent Framework for Explainable Automatic Differential Diagnosis
Daniel Rose, Chia-Chien Hung, Marco Lepri +3
Differential Diagnosis (DDx) is a fundamental yet complex aspect of clinical decision-making, in which physicians iteratively refine a ranked list of possible diseases based on sym…
Evaluating Language Models as Synthetic Data Generators
Seungone Kim, Juyoung Suk, Xiang Yue +7
Given the increasing use of synthetic data in language model (LM) post-training, an LM's ability to generate high-quality data has become nearly as crucial as its ability to solve…
Aligning Generalisation Between Humans and Machines
Filip Ilievski, Barbara Hammer, Frank van Harmelen +22
Recent advances in AI -- including generative approaches -- have resulted in technology that can support humans in scientific discovery and forming decisions, but may also disrupt…
LightPAL: Lightweight Passage Retrieval for Open Domain Multi-Document Summarization
Masafumi Enomoto, Kunihiro Takeoka, Kosuke Akimoto +2
Open-Domain Multi-Document Summarization (ODMDS) is the task of generating summaries from large document collections in response to user queries. This task is crucial for efficient…
AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agents
Luca Gioacchini, Giuseppe Siracusano, Davide Sanvito +4
The advances made by Large Language Models (LLMs) have led to the pursuit of LLM agents that can solve intricate, multi-step reasoning tasks. As with any research pursuit, benchmar…
Robust Text Classification: Analyzing Prototype-Based Networks
Zhivar Sourati, Darshan Deshpande, Filip Ilievski +2
Downstream applications often require text classification models to be accurate and robust. While the accuracy of the state-of-the-art Language Models (LMs) approximates human perf…
Linking Surface Facts to Large-Scale Knowledge Graphs
Gorjan Radevski, Kiril Gashteovski, Chia-Chien Hung +2
Open Information Extraction (OIE) methods extract facts from natural language text in the form of ("subject"; "relation"; "object") triples. These facts are, however, merely surfac…
Large Language Models Enable Few-Shot Clustering
Vijay Viswanathan, Kiril Gashteovski, Carolin Lawrence +2
Unlike traditional unsupervised clustering, semi-supervised clustering allows users to provide meaningful structure to the data, which helps the clustering algorithm to match the u…
Leveraging Open Information Extraction for More Robust Domain Transfer of Event Trigger Detection
David Dukić, Kiril Gashteovski, Goran Glavaš +1
Event detection is a crucial information extraction task in many domains, such as Wikipedia or news. The task typically relies on trigger detection (TD) -- identifying token spans…
KGxBoard: Explainable and Interactive Leaderboard for Evaluation of Knowledge Graph Completion Models
Haris Widjaja, Kiril Gashteovski, Wiem Ben Rim +5
Knowledge Graphs (KGs) store information in the form of (head, predicate, tail)-triples. To augment KGs with new knowledge, researchers proposed models for KG Completion (KGC) task…
Human-Centric Research for NLP: Towards a Definition and Guiding Questions
Bhushan Kotnis, Kiril Gashteovski, Julia Gastinger +7
With Human-Centric Research (HCR) we can steer research activities so that the research outcome is beneficial for human stakeholders, such as end users. But what exactly makes rese…
A Human-Centric Assessment Framework for AI
Sascha Saralajew, Ammar Shaker, Zhao Xu +5
With the rise of AI systems in real-world applications comes the need for reliable and trustworthy AI. An essential aspect of this are explainable AI systems. However, there is no…
milIE: Modular & Iterative Multilingual Open Information Extraction
Bhushan Kotnis, Kiril Gashteovski, Daniel Oñoro Rubio +5
Open Information Extraction (OpenIE) is the task of extracting (subject, predicate, object) triples from natural language sentences. Current OpenIE systems extract all triple slots…
AnnIE: An Annotation Platform for Constructing Complete Open Information Extraction Benchmark
Niklas Friedrich, Kiril Gashteovski, Mingying Yu +4
Open Information Extraction (OIE) is the task of extracting facts from sentences in the form of relations and their corresponding arguments in schema-free manner. Intrinsic perform…
BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation
Kiril Gashteovski, Mingying Yu, Bhushan Kotnis +3
Intrinsic evaluations of OIE systems are carried out either manually -- with human evaluators judging the correctness of extractions -- or automatically, on standardized benchmarks…
OPIEC: An Open Information Extraction Corpus
Kiril Gashteovski, Sebastian Wanner, Sven Hertling +2
Open information extraction (OIE) systems extract relations and their arguments from natural language text in an unsupervised manner. The resulting extractions are a valuable resou…