14 citations · 16 across the 11 of their papers we have counts for
14 papers
Audible World Models: Spatially Aware Sound Generation for 3D Worlds
Duowen Chen, Jinjin He, Gouthaman KV +2
Text- and image-conditioned world generators can create visually rich 3D environments, yet these worlds often remain silent or rely on soundtracks synthesized solely from text or r…
Multichannel Audio Quality Assessment: Extending Pretrained Perceptual Models to Spatial Audio
Gouthaman KV, Shiv Gehlot, Vishnu Raj +2
Accurate perceptual quality assessment is essential for evaluating and optimizing spatial audio, where perceived quality depends on both signal fidelity and inter-channel spatial r…
Spot, Separate, and Enhance: Fully Generative Approach for Audio Mixing
Ilpo Viertola, Giulio Cengarle, Gouthaman KV +2
We introduce Spot, Separate, and Enhance (SSE), the first multimodal, user-guided generative model for audio remixing and enhancement. SSE enhances video content by rebalancing the…
VIBE: Video Instruction-aligned Background music gEneration
Aryan Vijay Bhosale, Vaibhavi Lokegaonkar, Vishnu Raj +5
Current video-to-music (V2M) models lack semantic control and fail to penalize instruction violations, largely due to their reliance on reconstruction objectives and the representa…
A robust PPG foundation model using multimodal physiological supervision
Eloy Geenjaar, Vince Calhoun, Scott Daly +4
Photoplethysmography (PPG), a non-invasive measure of changes in blood volume, is widely used in both wearable devices and clinical settings. Recent PPG foundation models either us…
Video-Robin: Autoregressive Diffusion Planning for Intent-Grounded Video-to-Music Generation
Vaibhavi Lokegaonkar, Aryan Vijay Bhosale, Vishnu Raj +5
Video-to-music (V2M) is the fundamental task of creating background music for an input video. Recent V2M models achieve audiovisual alignment by typically relying on visual conditi…
MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Models
Xiyang Wu, Zongxia Li, Jihui Jin +7
Vision Language Models (VLMs) perform well on standard video tasks but struggle with physics-related reasoning involving motion dynamics and spatial interactions. We present a nove…
Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddings
Aakriti Agrawal, Gouthaman KV, Rohith Aralikatti +6
Hallucinations in Large Vision-Language Models (LVLMs) remain a persistent challenge, often stemming from inadequate integration of visual information during multimodal reasoning.…
Enhanced Generative Machine Listener
Vishnu Raj, Gouthaman KV, Shiv Gehlot +2
We present GMLv2, a reference-based model designed for the prediction of subjective audio quality as measured by MUSHRA scores. GMLv2 introduces a Beta distribution-based loss to m…
Moment Sampling in Video LLMs for Long-Form Video QA
Mustafa Chasmai, Gauri Jagatap, Gouthaman KV +3
Recent advancements in video large language models (Video LLMs) have significantly advanced the field of video question answering (VideoQA). While existing methods perform well on…
Semi-Supervised Contrastive Learning for Controllable Video-to-Music Retrieval
Shanti Stewart, Gouthaman KV, Lie Lu +1
Content creators often use music to enhance their videos, from soundtracks in movies to background music in video blogs and social media content. However, identifying the best musi…
On the Significance of Question Encoder Sequence Model in the Out-of-Distribution Performance in Visual Question Answering
Gouthaman KV, Anurag Mittal
Generalizing beyond the experiences has a significant role in developing practical AI systems. It has been shown that current Visual Question Answering (VQA) models are over-depend…
Linguistically-aware Attention for Reducing the Semantic-Gap in Vision-Language Tasks
Gouthaman KV, Athira Nambiar, Kancheti Sai Srinivas +1
Attention models are widely used in Vision-language (V-L) tasks to perform the visual-textual correlation. Humans perform such a correlation with a strong linguistic understanding…
Reducing Language Biases in Visual Question Answering with Visually-Grounded Question Encoder
Gouthaman KV, Anurag Mittal
Recent studies have shown that current VQA models are heavily biased on the language priors in the train set to answer the question, irrespective of the image. E.g., overwhelmingly…