566 citations · 699 across the 10 of their papers we have counts for
6 papers · 1 filter
MJEPA: A Simple and Scalable Joint-Embedding Predictive Architecture for Audio-Visual Learning
Revant Teotia, Adrien Bardes, Michael Rabbat +3
Self-supervised learning from large-scale video data has emerged as a dominant paradigm for visual representation learning. Since audio and visual streams naturally co-occur in vid…
Characterizing the Predictive Impact of Modalities with Supervised Latent-Variable Modeling
Divyam Madaan, Sumit Chopra, Kyunghyun Cho
Despite the recent success of Multimodal Large Language Models (MLLMs), existing approaches predominantly assume the availability of multiple modalities during training and inferen…
Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensional
Divyam Madaan, Varshan Muhunthan, Kyunghyun Cho +1
Understanding the interplay between intra-modality dependencies (the contribution of an individual modality to a target task) and inter-modality dependencies (the relationships bet…
DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Models
Revant Teotia, Candace Ross, Karen Ullrich +4
Recent advances in text-to-image (T2I) models have achieved impressive quality and consistency. However, this has come at the cost of representation diversity. While automatic eval…
A training regime to learn unified representations from complementary breast imaging modalities
Umang Sharma, Jungkyu Park, Laura Heacock +2
Full Field Digital Mammograms (FFDMs) and Digital Breast Tomosynthesis (DBT) are the two most widely used imaging modalities for breast cancer screening. Although DBT has increased…
Jointly Modeling Inter- & Intra-Modality Dependencies for Multi-modal Learning
Divyam Madaan, Taro Makino, Sumit Chopra +1
Supervised multi-modal learning involves mapping multiple modalities to a target label. Previous studies in this field have concentrated on capturing in isolation either the inter-…