5 papers
AfterDeleteBench: Auditing Post-Deletion Information Persistence in LLM Agents
Ehsan Saleh, Roozbeh Nahavandi, Sama Salarian +2
Many LLM agents maintain external persistent memory and derive additional state from past interactions, such as summaries. Deleting an original memory record therefore need not rem…
RealityBench: Simulated Environments to Evaluate a Vision Language Model's Ability to Document Human Physical Work
Andrew Perrault, Anmol Takiar, Dominik Winecki +2
Augmenting worker capabilities in the physical world through guidance has been a decades-long goal for AI agents. One key supporting capability is to understand what the worker is…
Generating Synthetic Doctor-Patient Conversations for Long-form Audio Summarization
Yanis Labrak, David Grünert, Séverin Baroudi +11
Long-context audio reasoning is underserved in both training data and evaluation. Existing benchmarks target short-context tasks, and the open-ended generation tasks most relevant…
Beyond Length: Context-Aware Expansion and Independence as Developmentally Sensitive Evaluation in Child Utterances
Jiyun Chun, Eric Fosler-Lussier, Michael White +1
Evaluating the quality of children's utterances in adult-child dialogue remains challenging due to insufficient context-sensitive metrics. Common proxies such as Mean Length of Utt…
VISTA: Verification In Sequential Turn-based Assessment
Ashley Lewis, Andrew Perrault, Eric Fosler-Lussier +1
Hallucination--defined here as generating statements unsupported or contradicted by available evidence or conversational context--remains a major obstacle to deploying conversation…