EH-Benchmark Ophthalmic Hallucination Benchmark and Agent-Driven Top-Down Traceable Reasoning Workflow

Abstract

Medical Large Language Models (MLLMs) play a crucial role in ophthalmic diagnosis, holding significant potential to address vision-threatening diseases. However, their accuracy is constrained by hallucinations stemming from limited ophthalmic knowledge, insufficient visual localization and reasoning capabilities, and a scarcity of multimodal ophthalmic data, which collectively impede precise lesion detection and disease diagnosis. Furthermore, existing medical benchmarks fail to effectively evaluate various types of hallucinations or provide actionable solutions to mitigate them. To address the above challenges, we introduce EH-Benchmark, a novel ophthalmology benchmark designed to evaluate hallucinations in MLLMs. We categorize MLLMs' hallucinations based on specific tasks and error types into two primary classes: Visual Understanding and Logical Composition, each comprising multiple subclasses. Given that MLLMs predominantly rely on language-based reasoning rather than visual processing, we propose an agent-centric, three-phase framework, including the Knowledge-Level Retrieval stage, the Task-Level Case Studies stage, and the Result-Level Validation stage. Experimental results show that our multi-agent framework significantly mitigates both types of hallucinations, enhancing accuracy, interpretability, and reliability. Our project is available at https://github.com/ppxy1/EH-Benchmark.

9 figures, 5 tables. submit/6621751

References in corpus (21)

• DeepSeek-V3 Technical Report
• BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks
• A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferability
• LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day
• Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
• OCTDL: Optical Coherence Tomography Dataset for Image-Based Deep Learning Methods
• Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
• Qwen2.5-VL Technical Report
• ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models
• Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generation
• Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents
• Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
• HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale
• HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation
• Detecting and Evaluating Medical Hallucinations in Large Vision Language Models
• Embodied LLM Agents Learn to Cooperate in Organized Teams
• MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models
• MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context
• Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization
• MedSAM-U: Uncertainty-Guided Auto Multi-Prompt Adaptation for Reliable MedSAM
• Measuring the Measurers: Quality Evaluation of Hallucination Benchmarks for Large Vision-Language Models