3 citations · 5 across the 11 of their papers we have counts for
12 papers
Into the Unknown: Benchmarking AI agents on open scientific research problems
Seungone Kim, Jiayi Geng, Young-Jun Lee +25
Can AI agents carry out research on open scientific problems? We introduce Into the Unknown, a benchmark that provides an agent with a science paper along with a research direction…
On the limits and opportunities of AI reviewers: Reviewing the reviews of Nature-family papers with 45 expert scientists
Seungone Kim, Dongkeun Yoon, Kiril Gashteovski +55
With the advancement of AI capabilities, AI reviewers are beginning to be deployed in scientific peer review, yet their capability and credibility remain in question: many scientis…
Oblivion: Self-Adaptive Agentic Memory Control through Decay-Driven Activation
Ashish Rana, Chia-Chien Hung, Qumeng Sun +2
Human memory adapts through selective forgetting: experiences become less accessible over time but can be reactivated by reinforcement or contextual cues. In contrast, memory-augme…
Compositional Steering of Large Language Models with Steering Tokens
Gorjan Radevski, Kiril Gashteovski, Giwon Hong +2
Deploying LLMs in real-world applications requires controllable output that satisfies multiple desiderata at the same time. While existing work extensively addresses LLM steering f…
On Synthesizing Data for Context Attribution in Question Answering
Gorjan Radevski, Kiril Gashteovski, Shahbaz Syed +11
Question Answering (QA) accounts for a significant portion of LLM usage "in the wild". However, LLMs sometimes produce false or misleading responses, also known as "hallucinations"…
Scaling Evaluation-time Compute with Reasoning Models as Evaluators
Seungone Kim, Ian Wu, Jinu Lee +8
As language model (LM) outputs get more and more natural, it is becoming more difficult than ever to evaluate their quality. Simultaneously, increasing LMs' "thinking" time through…
Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions
Emmy Liu, Amanda Bertsch, Lintang Sutawika +9
Improvements in language model capabilities are often attributed to increasing model size or training data, but in some cases smaller models trained on curated data or with differe…
MEDDxAgent: A Unified Modular Agent Framework for Explainable Automatic Differential Diagnosis
Daniel Rose, Chia-Chien Hung, Marco Lepri +3
Differential Diagnosis (DDx) is a fundamental yet complex aspect of clinical decision-making, in which physicians iteratively refine a ranked list of possible diseases based on sym…
Evaluating Language Models as Synthetic Data Generators
Seungone Kim, Juyoung Suk, Xiang Yue +7
Given the increasing use of synthetic data in language model (LM) post-training, an LM's ability to generate high-quality data has become nearly as crucial as its ability to solve…
AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agents
Luca Gioacchini, Giuseppe Siracusano, Davide Sanvito +4
The advances made by Large Language Models (LLMs) have led to the pursuit of LLM agents that can solve intricate, multi-step reasoning tasks. As with any research pursuit, benchmar…
Walking a Tightrope -- Evaluating Large Language Models in High-Risk Domains
Chia-Chien Hung, Wiem Ben Rim, Lindsay Frost +2
High-risk domains pose unique challenges that require language models to provide accurate and safe responses. Despite the great success of large language models (LLMs), such as Cha…
Linking Surface Facts to Large-Scale Knowledge Graphs
Gorjan Radevski, Kiril Gashteovski, Chia-Chien Hung +2
Open Information Extraction (OIE) methods extract facts from natural language text in the form of ("subject"; "relation"; "object") triples. These facts are, however, merely surfac…