10 papers
MJEPA: A Simple and Scalable Joint-Embedding Predictive Architecture for Audio-Visual Learning
Revant Teotia, Adrien Bardes, Michael Rabbat +3
Self-supervised learning from large-scale video data has emerged as a dominant paradigm for visual representation learning. Since audio and visual streams naturally co-occur in vid…
L-TGVN: Leveraging Longitudinal Priors for Personalized Rapid MRI
Arda Atalık, Sumit Chopra, Daniel K. Sodickson
MRI provides excellent soft-tissue contrast without ionizing radiation, but long acquisition times increase patient discomfort while also raising exam costs and limiting scanner th…
Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensional
Divyam Madaan, Varshan Muhunthan, Kyunghyun Cho +1
Understanding the interplay between intra-modality dependencies (the contribution of an individual modality to a target task) and inter-modality dependencies (the relationships bet…
Characterizing the Predictive Impact of Modalities with Supervised Latent-Variable Modeling
Divyam Madaan, Sumit Chopra, Kyunghyun Cho
Despite the recent success of Multimodal Large Language Models (MLLMs), existing approaches predominantly assume the availability of multiple modalities during training and inferen…
Temporal Generalization: A Reality Check
Divyam Madaan, Sumit Chopra, Kyunghyun Cho
Machine learning (ML) models often struggle to maintain performance under distribution shifts, leading to inaccurate predictions on unseen future data. In this work, we investigate…
DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Models
Revant Teotia, Candace Ross, Karen Ullrich +4
Recent advances in text-to-image (T2I) models have achieved impressive quality and consistency. However, this has come at the cost of representation diversity. While automatic eval…