6 papers
CoLA: Cross-Modal Low-rank Adaptation for Multimodal Downstream Tasks
Wish Suharitdamrong, Tony Alex, Muhammad Awais +1
Foundation models have revolutionized AI, but adapting them efficiently for multimodal tasks, particularly in dual-stream architectures composed of unimodal encoders, such as DINO…
From Senses to Decisions: The Information Flow of Auditory and Visual Perception in Multimodal LLMs
Wish Suharitdamrong, Muhammad Awais, Xiatian Zhu +1
Multimodal Large Language Models (MLLMs) can listen and see, but how do audio and visual signals actually travel through the network to shape an answer? Despite their growing role…
PAL: Probing Audio Encoders via LLMs -- Audio Information Transfer into LLMs
Tony Alex, Wish Suharitdamrong, Sara Atito +4
Integration of audio perception into large language models (LLMs) is an emerging research area for enabling machine listening applications, yet efficient transfer of rich audio sem…
Few-Label Multimodal Modeling of SNP Variants and ECG Phenotypes Using Large Language Models for Cardiovascular Risk Stratification
Niranjana Arun Menon, Yulong Li, Iqra Farooq +3
Cardiovascular disease (CVD) risk stratification remains a major challenge due to its multifactorial nature and limited availability of high-quality labeled datasets. While genomic…
How Effectively Can Large Language Models Connect SNP Variants and ECG Phenotypes for Cardiovascular Risk Prediction?
Niranjana Arun Menon, Iqra Farooq, Yulong Li +4
Cardiovascular disease (CVD) prediction remains a tremendous challenge due to its multifactorial etiology and global burden of morbidity and mortality. Despite the growing availabi…
SSLAM: Enhancing Self-Supervised Models with Audio Mixtures for Polyphonic Soundscapes
Tony Alex, Sara Ahmed, Armin Mustafa +2
Self-supervised pre-trained audio networks have seen widespread adoption in real-world systems, particularly in multi-modal large language models. These networks are often employed…