6 papers · 1 filter
Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Reward
Shizhan Gong, Minda Hu, Qiyuan Zhang +2
Vision-language models (VLMs) have achieved remarkable success across diverse tasks. However, concerns about their trustworthiness persist, particularly regarding tendencies to lea…
Concepts from Representations: Post-hoc Concept Bottleneck Models via Sparse Decomposition of Visual Representations
Shizhan Gong, Xiaofan Zhang, Qi Dou
Deep learning has achieved remarkable success in image recognition, yet their inherent opacity poses challenges for deployment in critical domains. Concept-based interpretations ai…
Kernel-based Unsupervised Embedding Alignment for Enhanced Visual Representation in Vision-language Models
Shizhan Gong, Yankai Jiang, Qi Dou +1
Vision-language models, such as CLIP, have achieved significant success in aligning visual and textual representations, becoming essential components of many multi-modal large lang…
A Super-pixel-based Approach to the Stable Interpretation of Neural Networks
Shizhan Gong, Jingwei Zhang, Qi Dou +1
Saliency maps are widely used in the computer vision community for interpreting neural network classifiers. However, due to the randomness of training samples and optimization algo…
FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Models
Ruinan Jin, Zikang Xu, Yuan Zhong +4
The advent of foundation models (FMs) in healthcare offers unprecedented opportunities to enhance medical diagnostics through automated classification and segmentation tasks. Howev…
Weakly-supervised Medical Image Segmentation with Gaze Annotations
Yuan Zhong, Chenhui Tang, Yumeng Yang +6
Eye gaze that reveals human observational patterns has increasingly been incorporated into solutions for vision tasks. Despite recent explorations on leveraging gaze to aid deep ne…