5 papers
AuralSAM2: Enabling SAM2 Hear Through Pyramid Audio-Visual Feature Prompting
Yuyuan Liu, Yuanhong Chen, Chong Wang +6
Segment Anything Model 2 (SAM2) exhibits strong generalisation for promptable segmentation in video clips; however, its integration with the audio modality remains underexplored. E…
Meta-Learned Modality-Weighted Knowledge Distillation for Robust Multi-Modal Learning with Missing Data
Hu Wang, Salma Hassan, Yuyuan Liu +12
In multi-modal learning, some modalities are more influential than others, and their absence can have a significant impact on classification/segmentation accuracy. Addressing this…
Mixture of Gaussian-distributed Prototypes with Generative Modelling for Interpretable and Trustworthy Image Recognition
Chong Wang, Yuanhong Chen, Fengbei Liu +4
Prototypical-part methods, e.g., ProtoPNet, enhance interpretability in image recognition by linking predictions to training prototypes, thereby offering intuitive insights into th…
Cross- and Intra-image Prototypical Learning for Multi-label Disease Diagnosis and Interpretation
Chong Wang, Fengbei Liu, Yuanhong Chen +2
Recent advances in prototypical learning have shown remarkable potential to provide useful decision interpretations associating activation maps and predictions with class-specific…
Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modality
Hu Wang, Congbo Ma, Jianpeng Zhang +4
The problem of missing modalities is both critical and non-trivial to be handled in multi-modal models. It is common for multi-modal tasks that certain modalities contribute more c…