6 papers
Listen, Reason, and Segment: Aligning LALMs with Editorial Judgment for Media Chapterization
Tony Alex, Wish Suharitdamrong, Sara Atito +5
Large Audio Language Models (LALMs) have made rapid progress on standardized benchmarks, yet their deployment in practical media workflows, curation, archival indexing, and content…
From Senses to Decisions: The Information Flow of Auditory and Visual Perception in Multimodal LLMs
Wish Suharitdamrong, Muhammad Awais, Xiatian Zhu +1
Multimodal Large Language Models (MLLMs) can listen and see, but how do audio and visual signals actually travel through the network to shape an answer? Despite their growing role…
CoLA: Cross-Modal Low-rank Adaptation for Multimodal Downstream Tasks
Wish Suharitdamrong, Tony Alex, Muhammad Awais +1
Foundation models have revolutionized AI, but adapting them efficiently for multimodal tasks, particularly in dual-stream architectures composed of unimodal encoders, such as DINO…
Few-Label Multimodal Modeling of SNP Variants and ECG Phenotypes Using Large Language Models for Cardiovascular Risk Stratification
Niranjana Arun Menon, Yulong Li, Iqra Farooq +3
Cardiovascular disease (CVD) risk stratification remains a major challenge due to its multifactorial nature and limited availability of high-quality labeled datasets. While genomic…
SSLAM: Enhancing Self-Supervised Models with Audio Mixtures for Polyphonic Soundscapes
Tony Alex, Sara Ahmed, Armin Mustafa +2
Self-supervised pre-trained audio networks have seen widespread adoption in real-world systems, particularly in multi-modal large language models. These networks are often employed…
PAL: Probing Audio Encoders via LLMs -- Audio Information Transfer into LLMs
Tony Alex, Wish Suharitdamrong, Sara Atito +4
Integration of audio perception into large language models (LLMs) is an emerging research area for enabling machine listening applications, yet efficient transfer of rich audio sem…