6 papers
Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Reward
Shizhan Gong, Minda Hu, Qiyuan Zhang +2
Vision-language models (VLMs) have achieved remarkable success across diverse tasks. However, concerns about their trustworthiness persist, particularly regarding tendencies to lea…
Concepts from Representations: Post-hoc Concept Bottleneck Models via Sparse Decomposition of Visual Representations
Shizhan Gong, Xiaofan Zhang, Qi Dou
Deep learning has achieved remarkable success in image recognition, yet their inherent opacity poses challenges for deployment in critical domains. Concept-based interpretations ai…
Kernel-based Unsupervised Embedding Alignment for Enhanced Visual Representation in Vision-language Models
Shizhan Gong, Yankai Jiang, Qi Dou +1
Vision-language models, such as CLIP, have achieved significant success in aligning visual and textual representations, becoming essential components of many multi-modal large lang…
pFedFair: Towards Optimal Group Fairness-Accuracy Trade-off in Heterogeneous Federated Learning
Haoyu Lei, Shizhan Gong, Qi Dou +1
Federated learning (FL) algorithms commonly aim to maximize clients' accuracy by training a model on their collective data. However, in several FL applications, the model's decisio…
A Super-pixel-based Approach to the Stable Interpretation of Neural Networks
Shizhan Gong, Jingwei Zhang, Qi Dou +1
Saliency maps are widely used in the computer vision community for interpreting neural network classifiers. However, due to the randomness of training samples and optimization algo…
FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Models
Ruinan Jin, Zikang Xu, Yuan Zhong +4
The advent of foundation models (FMs) in healthcare offers unprecedented opportunities to enhance medical diagnostics through automated classification and segmentation tasks. Howev…