8 papers · 1 filter
Co-Learning for Missing Arbitrary Modalities in Multi-modal Classification
Francisco Mena, Dino Ienco, Roberto Interdonato +2
Multi-modal classification leverages complementary information across diverse data sources to enhance predictive performance. However, real-world scenarios subject to operational c…
Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations
Jonas Klotz, Cassio F. Dantas, Pallavi Jain +2
Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy met…
Metonymy in vision models undermines attention-based interpretability
Ananthu Aniraj, Cassio F. Dantas, Dino Ienco +2
Part-based reasoning is a classical strategy to make a computer vision model directly focus on the object parts that are relevant to the downstream task. In the context of deep lea…
Two-stage Vision Transformers and Hard Masking offer Robust Object Representations
Ananthu Aniraj, Cassio F. Dantas, Dino Ienco +1
Context can strongly affect object representations, sometimes leading to undesired biases, particularly when objects appear in out-of-distribution backgrounds at inference. At the…
Multi-modal Co-learning for Earth Observation: Enhancing single-modality models via modality collaboration
Francisco Mena, Dino Ienco, Cassio F. Dantas +2
Multi-modal co-learning is emerging as an effective paradigm in machine learning, enabling models to collaboratively learn from different modalities to enhance single-modality pred…
Revisiting Cross-Modal Knowledge Distillation: A Disentanglement Approach for RGBD Semantic Segmentation
Roger Ferrod, Cássio F. Dantas, Luigi Di Caro +1
Multi-modal RGB and Depth (RGBD) data are predominant in many domains such as robotics, autonomous driving and remote sensing. The combination of these multi-modal data enhances en…