8 papers
SynCo: Learning Cross-Modal Synergy by Contrasting Interaction Residuals
Yavuz Yarici, Ghassan AlRegib
Multimodal contrastive learning is a dominant paradigm for learning transferable representations from unlabeled data, but standard objectives primarily capture information that is…
MER-DG: Modality-Entropy Regularization for Multimodal Domain Generalization
Yavuz Yarici, Ghassan AlRegib
Deploying multimodal models in real-world scenarios requires generalization to new environments where recording conditions differ from training, a challenge known as multimodal dom…
Subject Invariant Contrastive Learning for Human Activity Recognition
Yavuz Yarici, Kiran Kokilepersaud, Mohit Prabhushankar +1
The high cost of annotating data makes self-supervised approaches, such as contrastive learning methods, appealing for Human Activity Recognition (HAR). Effective contrastive learn…
Hierarchical and Multimodal Data for Daily Activity Understanding
Ghazal Kaviani, Yavuz Yarici, Seulgi Kim +4
Daily Activity Recordings for Artificial Intelligence (DARai, pronounced "Dahr-ree") is a multimodal, hierarchically annotated dataset constructed to understand human activities in…
CRACKS: Crowdsourcing Resources for Analysis and Categorization of Key Subsurface faults
Mohit Prabhushankar, Kiran Kokilepersaud, Jorge Quesada +6
Crowdsourcing annotations has created a paradigm shift in the availability of labeled data for machine learning. Availability of large datasets has accelerated progress in common k…
Explaining Representation Learning with Perceptual Components
Yavuz Yarici, Kiran Kokilepersaud, Mohit Prabhushankar +1
Self-supervised models create representation spaces that lack clear semantic meaning. This interpretability problem of representations makes traditional explainability methods inef…
Taxes Are All You Need: Integration of Taxonomical Hierarchy Relationships into the Contrastive Loss
Kiran Kokilepersaud, Yavuz Yarici, Mohit Prabhushankar +1
In this work, we propose a novel supervised contrastive loss that enables the integration of taxonomic hierarchy information during the representation learning process. A supervise…
Exploiting the Distortion-Semantic Interaction in Fisheye Data
Kiran Kokilepersaud, Mohit Prabhushankar, Yavuz Yarici +2
In this work, we present a methodology to shape a fisheye-specific representation space that reflects the interaction between distortion and semantic context present in this data m…