ClinLens: Towards Long-Horizon LLM Agents for Longitudinal Multimodal Clinical Data Science

Abstract

Automating clinical data science requires agents to translate patient-centric objectives into executable workflows grounded in longitudinal multimodal evidence. Existing benchmarks provide limited coverage of two essential capabilities: grounding analyses in the correct patient episode and harmonizing multimodal observations. We introduce ClinLens, a patient-centric benchmark for evaluating large language model (LLM) agents on long-horizon clinical data science workflows. ClinLens comprises 126 executable tasks spanning five data modalities, four analytical scopes, and five analytical capabilities. The tasks assess agents' ability to maintain a coherent patient-specific analytical context across interdependent stages of evidence grounding, multimodal integration, and analytical pipeline generation and execution. We evaluate general-purpose, coding, and biomedical agents, with task runs averaging 36.93 steps and 8.06 minutes. The strongest evaluated agent achieves an execution success rate of 86.4% and a final-answer accuracy of 54.0%, but a strict pass rate of only 29.4%. Failure analysis identifies patient-episode grounding as the most frequently recorded failure category under the evaluation procedure. By jointly evaluating analytical outcomes and the validity of the supporting clinical context, ClinLens provides a testbed for advancing reliable agents for longitudinal clinical data science.