6 papers
Re-M3Dr: Rebalanced MultiModal Mean Deviation Regression
Haojie Yin, Chengcheng Feng, Tianyi Liu +2
Mean Deviation (MD) is a critical metric for assessing visual field loss in ophthalmology. While previous work has focused solely on predicting MD from Optical Coherence Tomography…
DAOS: A Multimodal In-cabin Behavior Monitoring with Driver Action-Object Synergy Dataset
Yiming Li, Chen Cai, Tianyi Liu +5
In driver activity monitoring, movements are mostly limited to the upper body, which makes many actions look similar. To tell these actions apart, human often rely on the objects t…
Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representation
Zhaorui Tan, Xi Yang, Tan Pan +8
Variations in medical imaging modalities and individual anatomical differences pose challenges to cross-modality generalization in multi-modal tasks. Existing methods often concent…
EVOKE: Elevating Chest X-ray Report Generation via Multi-View Contrastive Learning and Patient-Specific Knowledge
Qiguang Miao, Kang Liu, Zhuoqi Ma +6
Radiology reports are crucial for planning treatment strategies and facilitating effective doctor-patient communication. However, the manual creation of these reports places a sign…
MedMAP: Promoting Incomplete Multi-modal Brain Tumor Segmentation with Alignment
Tianyi Liu, Zhaorui Tan, Muyin Chen +3
Brain tumor segmentation is often based on multiple magnetic resonance imaging (MRI). However, in clinical practice, certain modalities of MRI may be missing, which presents a more…
Multimodal Guidance Network for Missing-Modality Inference in Content Moderation
Zhuokai Zhao, Harish Palani, Tianyi Liu +2
Multimodal deep learning, especially vision-language models, have gained significant traction in recent years, greatly improving performance on many downstream tasks, including con…