6 papers
Information Router for Mitigating Modality Dominance in Vision-Language Models
Seulgi Kim, Mohit Prabhushankar, Ghassan AlRegib
Vision Language models (VLMs) have demonstrated strong performance across a wide range of benchmarks, yet they often suffer from modality dominance, where predictions rely dispropo…
A Novel Automatic Framework for Speaker Drift Detection in Synthesized Speech
Jia-Hong Huang, Seulgi Kim, Yi Chieh Liu +5
Recent diffusion-based text-to-speech (TTS) models achieve high naturalness and expressiveness, yet often suffer from speaker drift, a subtle, gradual shift in perceived speaker id…
Countering Multi-modal Representation Collapse through Rank-targeted Fusion
Seulgi Kim, Kiran Kokilepersaud, Mohit Prabhushankar +1
Multi-modal fusion methods often suffer from two types of representation collapse: feature collapse where individual dimensions lose their discriminative power (as measured by eige…
Multi-level and Multi-modal Action Anticipation
Seulgi Kim, Ghazal Kaviani, Mohit Prabhushankar +1
Action anticipation, the task of predicting future actions from partially observed videos, is crucial for advancing intelligent systems. Unlike action recognition, which operates o…
Hierarchical and Multimodal Data for Daily Activity Understanding
Ghazal Kaviani, Yavuz Yarici, Seulgi Kim +4
Daily Activity Recordings for Artificial Intelligence (DARai, pronounced "Dahr-ree") is a multimodal, hierarchically annotated dataset constructed to understand human activities in…
HEX: Hierarchical Emergence Exploitation in Self-Supervised Algorithms
Kiran Kokilepersaud, Seulgi Kim, Mohit Prabhushankar +1
In this paper, we propose an algorithm that can be used on top of a wide variety of self-supervised (SSL) approaches to take advantage of hierarchical structures that emerge during…