4 papers
WorkWorlds: An Infrastructure for Evaluating AI Agents on Workplace Tasks
Yining Hua, Levi Lian
Many knowledge-work benchmarks are constructed around individual tasks, with the context needed for each task selected together with or after the task has been specified. This desi…
StagedWorkspace: A Versioned Workspace for Knowledge-Work Agents
Yining Hua, Hongbin Na, Yifan Zhou +3
AI agents increasingly perform knowledge work (i.e., produce and modify persistent digital artifacts such as code repositories, documents, spreadsheets, slides, reports), yet the p…
LLMs for Medical Consultation Are Evaluated Too Late: The Preformulation Gap
Yining Hua, Cyrus Ayubcha, Hongbin Na +4
Large language models for medical consultation are often evaluated after a clinical problem has already been made clear, although real consultations may begin with a vague, minimiz…
Designing Benchmarks for Knowledge Work
Yining Hua, Hongbin Na, Cyrus Ayubcha +1
AI agents are moving quickly from answering isolated questions toward completing work through tools, software environments, and multi-step workflows. Much of what these systems are…