8 papers · 1 filter
Improving Medical Visual Reinforcement Fine-Tuning via Perception and Reasoning Augmentation
Guangjing Yang, ZhangYuan Yu, Ziyuan Qin +7
While recent advances in Reinforcement Fine-Tuning (RFT) have shown that rule-based reward schemes can enable effective post-training for large language models, their extension to…
Evaluating the Diagnostic Classification Ability of Multimodal Large Language Models: Insights from the Osteoarthritis Initiative
Li Wang, Xi Chen, XiangWen Deng +4
Multimodal large language models (MLLMs) show promising performance on medical visual question answering (VQA) and report generation, but these generation and explanation abilities…
iDPA: Instance Decoupled Prompt Attention for Incremental Medical Object Detection
Huahui Yi, Wei Xu, Ziyuan Qin +4
Existing prompt-based approaches have demonstrated impressive performance in continual learning, leveraging pre-trained large-scale models for classification tasks; however, the ti…
Guiding Medical Vision-Language Models with Explicit Visual Prompts: Framework Design and Comprehensive Exploration of Prompt Variations
Kangyu Zhu, Ziyuan Qin, Huahui Yi +4
While mainstream vision-language models (VLMs) have advanced rapidly in understanding image level information, they still lack the ability to focus on specific areas designated by…
Evaluating Hallucination in Text-to-Image Diffusion Models with Scene-Graph based Question-Answering Agent
Ziyuan Qin, Dongjie Cheng, Haoyu Wang +5
Contemporary Text-to-Image (T2I) models frequently depend on qualitative human evaluations to assess the consistency between synthesized images and the text prompts. There is a dem…
Unified Modeling Enhanced Multimodal Learning for Precision Neuro-Oncology
Huahui Yi, Xiaofei Wang, Kang Li +1
Multimodal learning, integrating histology images and genomics, promises to enhance precision oncology with comprehensive views at microscopic and molecular levels. However, existi…