19 citations · 137 across the 64 of their papers we have counts for
22 papers · 1 filter
SEA-LM: Egocentric Spatial Audio Understanding for Wearable Microphone Arrays
Sonal Kumar, Sinan Hersek, Artem Dementyev +6
Embodied, ego-centric intelligence fundamentally requires the ability to comprehend spatial audio within complex environments. While large audio-language models excel at mono-chann…
ParA-LLM: A Unified Approach to Paralinguistic and Acoustic Speech Understanding
Nishit Anand, Jiaqi Su, Ke Chen +5
Recent advances in Audio LLMs have achieved human-level speech recognition, yet existing systems struggle to capture paralinguistic aspects such as speaker traits, expressive varia…
VIBE: Video Instruction-aligned Background music gEneration
Aryan Vijay Bhosale, Vaibhavi Lokegaonkar, Vishnu Raj +5
Current video-to-music (V2M) models lack semantic control and fail to penalize instruction violations, largely due to their reliance on reconstruction objectives and the representa…
TEMPO: Temporally-grounded Multi-task Post-training for Large Audio-Language Models
Apoorva Kulkarni, Kaousheik Jayakumar, Sreyan Ghosh +3
Large audio-language models (LALMs) describe audio at the clip level but cannot assign timestamps to the events, speakers, or sounds they identify. Despite being essential for down…
A Closer Look at Failure Modes in Temporal Understanding of Large Audio-Language Models
Apoorva Kulkarni, Kaousheik Jayakumar, Sreyan Ghosh +3
Large Audio Language Models (LALMs) achieve strong performance on a variety of audio understanding tasks but continue to struggle with temporal reasoning, a fundamental capability…
FIGMA: Towards FIne-Grained Music retrievAl
Nishit Anand, Ashish Seth, Sreyan Ghosh +2
Retrieving music using natural language descriptions has improved with contrastive audio-text models such as CLAP, but current systems remain limited to coarse semantic queries. Wh…
Video-Robin: Autoregressive Diffusion Planning for Intent-Grounded Video-to-Music Generation
Vaibhavi Lokegaonkar, Aryan Vijay Bhosale, Vishnu Raj +5
Video-to-music (V2M) is the fundamental task of creating background music for an input video. Recent V2M models achieve audiovisual alignment by typically relying on visual conditi…
Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music
Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar +15
We present Audio Flamingo Next (AF-Next), the next-generation and most capable large audio-language model in the Audio Flamingo series, designed to advance understanding and reason…
Audio Hallucination Attacks: Probing the Reliability of Large Audio Language Models
Ashish Seth, Sonal Kumar, Ramaneswaran Selvakumar +5
Large Audio Language Models (LALMs) achieve strong performance on audio-language tasks; however, their reliability in real-world settings remains underexplored. We introduce Audio…
Analytical Exploration of Spatial Audio Cues: A Differentiable Multi-Sphere Scattering Model
Siminfar Samakoush Galougah, Pranav Pulijala, Ramani Duraiswami
A primary challenge in developing synthetic spatial hearing systems, particularly underwater, is accurately modeling sound scattering. Biological organisms achieve 3D spatial heari…
ORCA: Open-ended Response Correctness Assessment for Audio Question Answering
Šimon Sedláček, Sara Barahona, Bolaji Yusuf +9
Reliable assessment of the abilities of large audio language models (LALMs) is essential to advancing the state of the art. As benchmarks rapidly evolve to incorporate complex reas…
Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models
Arushi Goel, Sreyan Ghosh, Jaehyeon Kim +8
We present Audio Flamingo 3 (AF3), a fully open state-of-the-art (SOTA) large audio-language model that advances reasoning and understanding across speech, sound, and music. AF3 in…
Multi-Domain Audio Question Answering Benchmark Toward Acoustic Content Reasoning
Chao-Han Huck Yang, Sreyan Ghosh, Qing Wang +14
We present Task 5 of the DCASE 2025 Challenge: an Audio Question Answering (AQA) benchmark spanning multiple domains of sound understanding. This task defines three QA subsets (Bio…
TSPE: Task-Specific Prompt Ensemble for Improved Zero-Shot Audio Classification
Nishit Anand, Ashish Seth, Ramani Duraiswami +1
Audio-language models (ALMs) excel in zero-shot audio classification, a task where models classify previously unseen audio clips at test time by leveraging descriptive natural lang…
Differentiable Optimization of Linear Differential Microphone Arrays: A Joint Geometry and Filter Design Framework
Siminfar Samakoush Galougah, Ramani Duraiswami
This paper presents a differentiable optimization framework for the design of constrained Linear Differential Microphone Arrays (LDMAs). The proposed method leverages a non-uniform…
Biomimetic Frontend for Differentiable Audio Processing
Ruolan Leslie Famularo, Dmitry N. Zotkin, Shihab A. Shamma +1
While models in audio and speech processing are becoming deeper and more end-to-end, they as a consequence need expensive training on large data, and are often brittle. We build on…
GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities
Sreyan Ghosh, Sonal Kumar, Ashish Seth +6
Perceiving and understanding non-speech sounds and non-verbal speech is essential to making decisions that help us interact with our surroundings. In this paper, we propose GAMA, a…
CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models
Sreyan Ghosh, Ashish Seth, Sonal Kumar +7
A fundamental characteristic of audio is its compositional nature. Audio-language models (ALMs) trained using a contrastive approach (e.g., CLAP) that learns a shared representatio…
Multiple scattering ambisonics: three-dimensional sound field estimation using interacting spheres
Shoken Kaneko, Ramani Duraiswami
Rigid spherical microphone arrays (RSMAs) have been widely used in ambisonics sound field recording. While it is desired to combine the information captured by a grid of densely ar…
Spheroidal Ambisonics: a Spatial Audio Framework Using Spheroidal Bases
Shoken Kaneko
Ambisonics is an established framework to capture, process, and reproduce spatial sound fields based on its spherical harmonics representation. We propose a generalization of conve…
Gaussian Process Models for HRTF based Sound-Source Localization and Active-Learning
Yuancheng Luo, Dmitry N. Zotkin, Ramani Duraiswami
From a machine learning perspective, the human ability localize sounds can be modeled as a non-parametric and non-linear regression problem between binaural spectral features of so…
Sparse Head-Related Impulse Response for Efficient Direct Convolution
Yuancheng Luo, Dmitry N. Zotkin, Ramani Duraiswami
Head-related impulse responses (HRIRs) are subject-dependent and direction-dependent filters used in spatial audio synthesis. They describe the scattering response of the head, tor…