5 papers · 1 filter
Information Router for Mitigating Modality Dominance in Vision-Language Models
Seulgi Kim, Mohit Prabhushankar, Ghassan AlRegib
Vision Language models (VLMs) have demonstrated strong performance across a wide range of benchmarks, yet they often suffer from modality dominance, where predictions rely dispropo…
Hierarchical and Multimodal Data for Daily Activity Understanding
Ghazal Kaviani, Yavuz Yarici, Seulgi Kim +4
Daily Activity Recordings for Artificial Intelligence (DARai, pronounced "Dahr-ree") is a multimodal, hierarchically annotated dataset constructed to understand human activities in…
Countering Multi-modal Representation Collapse through Rank-targeted Fusion
Seulgi Kim, Kiran Kokilepersaud, Mohit Prabhushankar +1
Multi-modal fusion methods often suffer from two types of representation collapse: feature collapse where individual dimensions lose their discriminative power (as measured by eige…
Multi-level and Multi-modal Action Anticipation
Seulgi Kim, Ghazal Kaviani, Mohit Prabhushankar +1
Action anticipation, the task of predicting future actions from partially observed videos, is crucial for advancing intelligent systems. Unlike action recognition, which operates o…
HEX: Hierarchical Emergence Exploitation in Self-Supervised Algorithms
Kiran Kokilepersaud, Seulgi Kim, Mohit Prabhushankar +1
In this paper, we propose an algorithm that can be used on top of a wide variety of self-supervised (SSL) approaches to take advantage of hierarchical structures that emerge during…