8 papers
MOMO: Human Motion from Moving Sounds
Deli Cai, Shuyang Xu, Zhiyang Dou +4
Human responses to sound depend not only on its semantic content, but also on the evolving spatial relationship between source and listener, conveyed through spatial audio and its…
Enabling Immersive Audio-Visual Experience from Any Video
Zitong Lan, Mutian Tong, Jiatao Gu +1
Most videos capture only a narrow field of view and provide no spatial audio, limiting the sense of immersion they can provide. Recent video generation models can expand perspectiv…
EigeNet: Geometry-Informed Multi-Modal Learning for Few-shot Novel View RIR Prediction
Chong Jing, Zitong Lan, Junan Zhang +1
Predicting spatially varying Room Impulse Response (RIR) from sparse observations is a critical but highly challenging inverse problem for immersive spatial audio rendering. In thi…
Building Audio-Visual Digital Twins with Smartphones
Zitong Lan, Yiwei Tang, Yuhan Wang +3
Digital twins today are almost entirely visual, overlooking acoustics-a core component of spatial realism and interaction. We introduce AV-Twin, the first practical system that con…
Resounding Acoustic Fields with Reciprocity
Zitong Lan, Yiduo Hao, Mingmin Zhao
Achieving immersive auditory experiences in virtual environments requires flexible sound modeling that supports dynamic source positions. In this paper, we introduce a task called…
Guiding Audio Editing with Audio Language Model
Zitong Lan, Yiduo Hao, Mingmin Zhao
Audio editing plays a central role in VR/AR immersion, virtual conferencing, sound design, and other interactive media. However, recent generative audio editing models depend on te…
Acoustic Volume Rendering for Neural Impulse Response Fields
Zitong Lan, Chenhao Zheng, Zhiwei Zheng +1
Realistic audio synthesis that captures accurate acoustic phenomena is essential for creating immersive experiences in virtual and augmented reality. Synthesizing the sound receive…
Exploring quantum sensing for fine-grained liquid recognition
Yuechun Jiao, Jinlian Hu, Zitong Lan +9
Recent years have witnessed the use of pervasive wireless signals (e.g., Wi-Fi, RFID, and mmWave) for sensing purposes. Due to its non-intrusive characteristic, wireless sensing pl…