933 citations · 3k across the 27 of their papers we have counts for
31 papers
MJEPA: A Simple and Scalable Joint-Embedding Predictive Architecture for Audio-Visual Learning
Revant Teotia, Adrien Bardes, Michael Rabbat +3
Self-supervised learning from large-scale video data has emerged as a dominant paradigm for visual representation learning. Since audio and visual streams naturally co-occur in vid…
L-TGVN: Leveraging Longitudinal Priors for Personalized Rapid MRI
Arda Atalık, Sumit Chopra, Daniel K. Sodickson
MRI provides excellent soft-tissue contrast without ionizing radiation, but long acquisition times increase patient discomfort while also raising exam costs and limiting scanner th…
Characterizing the Predictive Impact of Modalities with Supervised Latent-Variable Modeling
Divyam Madaan, Sumit Chopra, Kyunghyun Cho
Despite the recent success of Multimodal Large Language Models (MLLMs), existing approaches predominantly assume the availability of multiple modalities during training and inferen…
Temporal Generalization: A Reality Check
Divyam Madaan, Sumit Chopra, Kyunghyun Cho
Machine learning (ML) models often struggle to maintain performance under distribution shifts, leading to inaccurate predictions on unseen future data. In this work, we investigate…
Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensional
Divyam Madaan, Varshan Muhunthan, Kyunghyun Cho +1
Understanding the interplay between intra-modality dependencies (the contribution of an individual modality to a target task) and inter-modality dependencies (the relationships bet…
DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Models
Revant Teotia, Candace Ross, Karen Ullrich +4
Recent advances in text-to-image (T2I) models have achieved impressive quality and consistency. However, this has come at the cost of representation diversity. While automatic eval…
A Trust-Guided Approach to MR Image Reconstruction with Side Information
Arda Atalık, Sumit Chopra, Daniel K. Sodickson
Reducing MRI scan times can improve patient care and lower healthcare costs. Many acceleration methods are designed to reconstruct diagnostic-quality images from sparse k-space dat…
HIST-AID: Leveraging Historical Patient Reports for Enhanced Multi-Modal Automatic Diagnosis
Haoxu Huang, Cem M. Deniz, Kyunghyun Cho +2
Chest X-ray imaging is a widely accessible and non-invasive diagnostic tool for detecting thoracic abnormalities. While numerous AI models assist radiologists in interpreting these…
Fine-Tuning In-House Large Language Models to Infer Differential Diagnosis from Radiology Reports
Luoyao Chen, Revant Teotia, Antonio Verdone +4
Radiology reports summarize key findings and differential diagnoses derived from medical imaging examinations. The extraction of differential diagnoses is crucial for downstream ta…
BURExtract-Llama: An LLM for Clinical Concept Extraction in Breast Ultrasound Reports
Yuxuan Chen, Haoyan Yang, Hengkai Pan +6
Breast ultrasound is essential for detecting and diagnosing abnormalities, with radiology reports summarizing key findings like lesion characteristics and malignancy assessments. E…
A training regime to learn unified representations from complementary breast imaging modalities
Umang Sharma, Jungkyu Park, Laura Heacock +2
Full Field Digital Mammograms (FFDMs) and Digital Breast Tomosynthesis (DBT) are the two most widely used imaging modalities for breast cancer screening. Although DBT has increased…
Adaptive Sampling of k-Space in Magnetic Resonance for Rapid Pathology Prediction
Chen-Yu Yen, Raghav Singhal, Umang Sharma +3
Magnetic Resonance (MR) imaging, despite its proven diagnostic utility, remains an inaccessible imaging modality for disease surveillance at the population level. A major factor re…
Jointly Modeling Inter- & Intra-Modality Dependencies for Multi-modal Learning
Divyam Madaan, Taro Makino, Sumit Chopra +1
Supervised multi-modal learning involves mapping multiple modalities to a target label. Previous studies in this field have concentrated on capturing in isolation either the inter-…
On Sensitivity and Robustness of Normalization Schemes to Input Distribution Shifts in Automatic MR Image Diagnosis
Divyam Madaan, Daniel Sodickson, Kyunghyun Cho +1
Magnetic Resonance Imaging (MRI) is considered the gold standard of medical imaging because of the excellent soft-tissue contrast exhibited in the images reconstructed by the MRI p…
FastMRI Prostate: A Publicly Available, Biparametric MRI Dataset to Advance Machine Learning for Prostate Cancer Imaging
Radhika Tibrewala, Tarun Dutt, Angela Tong +8
The fastMRI brain and knee dataset has enabled significant advances in exploring reconstruction methods for improving speed and image quality for Magnetic Resonance Imaging (MRI) v…
On the Feasibility of Machine Learning Augmented Magnetic Resonance for Point-of-Care Identification of Disease
Raghav Singhal, Mukund Sudarshan, Anish Mahishi +7
Early detection of many life-threatening diseases (e.g., prostate and breast cancer) within at-risk population can improve clinical outcomes and reduce cost of care. While numerous…
Generative Image Translation for Data Augmentation of Bone Lesion Pathology
Anant Gupta, Srivas Venkatesh, Sumit Chopra +1
Insufficient training data and severe class imbalance are often limiting factors when developing machine learning models for the classification of rare diseases. In this work, we a…
StarSpace: Embed All The Things!
Ledell Wu, Adam Fisch, Sumit Chopra +3
We present StarSpace, a general-purpose neural embedding model that can solve a wide variety of problems: labeling tasks such as text classification, ranking tasks such as informat…
Training Language Models Using Target-Propagation
Sam Wiseman, Sumit Chopra, Marc'Aurelio Ranzato +4
While Truncated Back-Propagation through Time (BPTT) is the most popular approach to training Recurrent Neural Networks (RNNs), it suffers from being inherently sequential (making…
Learning through Dialogue Interactions by Asking Questions
Jiwei Li, Alexander H. Miller, Sumit Chopra +2
A good dialogue agent should have the ability to interact with users by both responding to questions and by asking questions, and importantly to learn from both types of interactio…
Dialogue Learning With Human-In-The-Loop
Jiwei Li, Alexander H. Miller, Sumit Chopra +2
An important aspect of developing conversational agents is to give a bot the ability to improve through communicating with humans and to learn from the mistakes that it makes. Most…
Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systems
Jesse Dodge, Andreea Gane, Xiang Zhang +5
A long-term goal of machine learning is to build intelligent conversational agents. One recent popular approach is to train end-to-end models on a large amount of real dialog trans…
Sequence Level Training with Recurrent Neural Networks
Marc'Aurelio Ranzato, Sumit Chopra, Michael Auli +1
Many natural language processing applications use language models to generate text. These models are typically trained to predict the next word in a sequence, given the previous wo…
The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
Felix Hill, Antoine Bordes, Sumit Chopra +1
We introduce a new test of how well language models capture meaning in children's books. Unlike standard language modelling benchmarks, it distinguishes the task of predicting synt…