6 papers
SSAM: Singular Subspace Alignment for Merging Multimodal Large Language Models
Md Kaykobad Reza, Ameya Patil, Edward Ayrapetian +1
Multimodal large language models (MLLMs) achieve strong performance by jointly processing inputs from multiple modalities, such as vision, audio, and language. However, building su…
DualSwinFusionSeg: Multimodal Martian Landslide Segmentation via Dual Swin Transformer with Multi-Scale Fusion and UNet++
Shahriar Kabir, Abdullah Muhammed Amimul Ehsan, Istiak Ahmmed Rifti +1
Automated segmentation of Martian landslides, particularly in tectonically active regions such as Valles Marineris,is important for planetary geology, hazard assessment, and future…
Robust Multimodal Learning via Cross-Modal Proxy Tokens
Md Kaykobad Reza, Ameya Patil, Mashhour Solh +1
Multimodal models often experience a significant performance drop when one or more modalities are missing during inference. To address this challenge, we propose a simple yet effec…
Model, Analyze, and Comprehend User Interactions within a Social Media Platform
Md Kaykobad Reza, S M Maksudul Alam, Yiran Luo +2
In this study, we propose a novel graph-based approach to model, analyze and comprehend user interactions within a social media platform based on post-comment relationship. We cons…
MMP: Towards Robust Multi-Modal Learning with Masked Modality Projection
Niki Nezakati, Md Kaykobad Reza, Ameya Patil +2
Multimodal learning seeks to combine data from multiple input sources to enhance the performance of different downstream tasks. In real-world scenarios, performance can degrade sub…
Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation
Md Kaykobad Reza, Ashley Prater-Bennette, M. Salman Asif
Multimodal learning seeks to utilize data from multiple sources to improve the overall performance of downstream tasks. It is desirable for redundancies in the data to make multimo…