12 papers
Generalizing Multimodal Foundational Models in Context-Driven Class Incremental Learning
Avi Gupta, Nilotpal Sinha, Vishnu Raj +4
Class-Incremental Learning (CIL) aims to learn new classes continuously without forgetting the previously learned ones. Recent works in CIL have significantly advanced across vario…
Multichannel Audio Quality Assessment: Extending Pretrained Perceptual Models to Spatial Audio
Gouthaman KV, Shiv Gehlot, Vishnu Raj +2
Accurate perceptual quality assessment is essential for evaluating and optimizing spatial audio, where perceived quality depends on both signal fidelity and inter-channel spatial r…
Audio for Sports Highlight Detection: A Comparative Empirical Study
Hao Xu, Meenakshi Sarkar, Vishnu Raj +1
Sports highlight detection aims to identify the most exciting and meaningful moments from long sports videos. While existing methods often emphasize visual or visual-language repre…
VIBE: Video Instruction-aligned Background music gEneration
Aryan Vijay Bhosale, Vaibhavi Lokegaonkar, Vishnu Raj +5
Current video-to-music (V2M) models lack semantic control and fail to penalize instruction violations, largely due to their reliance on reconstruction objectives and the representa…
Listen, Look, and Learn: Learning Without Forgetting through SAM-Audio
Avi Gupta, Nilotpal Sinha, Vishnu Raj +4
Class-Incremental Learning (CIL) aims to continuously learn new classes without forgetting previously acquired knowledge. While recent CIL advances have spurred significant interes…
Video-Robin: Autoregressive Diffusion Planning for Intent-Grounded Video-to-Music Generation
Vaibhavi Lokegaonkar, Aryan Vijay Bhosale, Vishnu Raj +5
Video-to-music (V2M) is the fundamental task of creating background music for an input video. Recent V2M models achieve audiovisual alignment by typically relying on visual conditi…
Glow with the Flow: AI-Assisted Creation of Ambient Lightscapes for Music Videos
Frederic Anthony Robinson, Vishnu Raj, David Cooper +2
Designed light is an established modality for live performance and music playback. Despite the growing availability of consumer smart lighting, the creation of designed light for m…
DPAR: Dynamic Patchification for Efficient Autoregressive Visual Generation
Divyansh Srivastava, Akshay Mehra, Pranav Maneriker +5
Decoder-only autoregressive image generation typically relies on fixed-length tokenization schemes whose token counts grow quadratically with resolution, substantially increasing t…
CompanionCast: Toward Social Collaboration with Multi-Agent Systems in Shared Experiences
Yiyang Wang, Chen Chen, Tica Lin +4
Shared experiences are fundamental to social connection, yet media consumption is increasingly solitary. While AI companions offer real-time reactions and emotional regulation, exi…
MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Models
Xiyang Wu, Zongxia Li, Jihui Jin +7
Vision Language Models (VLMs) perform well on standard video tasks but struggle with physics-related reasoning involving motion dynamics and spatial interactions. We present a nove…
Event Camera Guided Visual Media Restoration & 3D Reconstruction: A Survey
Aupendu Kar, Vishnu Raj, Guan-Ming Su
Event camera sensors are bio-inspired sensors which asynchronously capture per-pixel brightness changes and output a stream of events encoding the polarity, location and time of th…
Enhanced Generative Machine Listener
Vishnu Raj, Gouthaman KV, Shiv Gehlot +2
We present GMLv2, a reference-based model designed for the prediction of subjective audio quality as measured by MUSHRA scores. GMLv2 introduces a Beta distribution-based loss to m…