5 papers
OmniEcho: Audio-Visual Spatial Understanding for Omni-Modal Embodied Agents
Ruixun Liu, Yuxuan Wang, Jiacheng Xie +10
Humans can effortlessly localize the direction of a sound source and integrate it with visual cues for reasoning, yet this remains challenging for embodied agents. In particular, i…
SHB-AE: Spherical harmonic beamforming based Ambisonics encoding and upscaling method for smartphone microphone array
Yuhuan You, Yufan Qian, Tianshu Qu +2
With the rapid development of virtual reality (VR) and augmented reality (AR), spatial audio recording and reproduction have gained increasing research interest. Higher Order Ambis…
Flow-HOA: Generative Joint Optimization for Ambisonics Encoding via Flow Matching
Yuhuan You, Yufan Qian, Tianshu Qu +2
Higher-Order Ambisonics (HOA) encoding from sparse, irregular microphone arrays remains a critical challenge for consumer spatial audio capture in immersive communication and XR. W…
The World is Not Mono: Enabling Spatial Understanding in Large Audio-Language Models
Yuhuan You, Lai Wei, Xihong Wu +1
Large audio-language models have made rapid progress in recognizing what is present in an audio clip, but spatial audio-language understanding still lacks a clear task interface. A…
TA-V2A: Textually Assisted Video-to-Audio Generation
Yuhuan You, Xihong Wu, Tianshu Qu
As artificial intelligence-generated content (AIGC) continues to evolve, video-to-audio (V2A) generation has emerged as a key area with promising applications in multimedia editing…