12 papers
KnowSim: Evaluating Information Calibration in LLM Assistants with User Simulators that Learn
Yoonjoo Lee, Hyoungwook Jin, Tae Soo Kim +3
To effectively collaborate with users on knowledge-intensive tasks, Large Language Models (LLMs) must perform information calibration: matching content to a user's evolving underst…
Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA
Sunah O, Jay-Yoon Lee
In multimodal multi-hop question answering, we focus on the initial retrieval stage via two distinct tasks: (1) evidence set completion, retrieving missing evidence given context,…
OpenAI GPT-5 System Card
Aaditya Singh, Adam Fry, Adam Perelman +483
This is the system card published alongside the OpenAI GPT-5 launch, August 2025. GPT-5 is a unified system with a smart and fast model that answers most questions, a deeper reason…
Precision Proactivity: Measuring Cognitive Load in Real-World AI-Assisted Work
Brandon Lepine, Juho Kim, Pamela Mishkin +1
Systems like ChatGPT and Claude assist billions through proactive dialogue-offering unsolicited, task-relevant information. Drawing on Cognitive Load Theory, we study how cognitive…
ConvApparel: A Benchmark Dataset and Validation Framework for User Simulators in Conversational Recommenders
Ofer Meshi, Krisztian Balog, Sally Goldman +5
The promise of LLM-based user simulators to improve conversational AI is hindered by a critical "realism gap," leading to systems that are optimized for simulated interactions, but…
DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culture
Arijit Maji, Raghvendra Kumar, Akash Ghosh +6
We introduce DRISHTIKON, a first-of-its-kind multimodal and multilingual benchmark centered exclusively on Indian culture, designed to evaluate the cultural understanding of genera…