2 citations · 4 across the 16 of their papers we have counts for
17 papers
One-Stage Multi-Task Instruction-Guided 3D Spatial Audio Editing
Ke Lei, Chenyuhao Wen, Yu Zhang +9
Spatial audio editing modifies an existing soundfield according to a user's instruction while preserving the rest of the scene. Unlike conventional audio editing, it must reason jo…
CSAVocoder: A Causal Spatial Audio Vocoder Towards Real-Time Spatial Audio Generation
Zhiyuan Zhu, Han Wang, Wenxiang Guo +4
Spatial audio vocoders are able to convert mel-spectrograms produced by generative models into spatial audio waveforms. Most neural vocoders are designed for monaural audio, and di…
VoxAudio: Vocalized Audio Synthesis via Multi-Reward Autoregressive Flow Matching
Wenxiang Guo, Changhao Pan, Ziyue Jiang +2
Vocalized audio synthesis, the task of generating audio in which intelligible speech is embedded within an environmental soundscape, underpins applications such as podcast producti…
Spatial-Omni: Spatial Audio Understanding Integration in Multimodal LLMs via FOA Encoding
Zhiyuan Zhu, Yixuan Chen, Yiwen Shao +13
Recent multimodal large language models mainly process audio as monaural signals, thereby discarding the spatial cues contained in spatial audio for sound localization, spatial rel…
Towards Streaming Synchronized Spatial Audio Generation via Autoregressive Diffusion Transformer
Ke Lei, Yu Zhang, Changhao Pan +4
Real-time and accurate spatial audio generation is pivotal for delivering an immersive experience. However, existing spatial audio synthesis technologies are often encumbered by a…
DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmark
Ruofan Hu, Menghui Zhu, Jieming Zhu +8
Multimodal documents contain diverse elements, such as tables, figures, and layouts, which can complicate retrieval tasks. While current approaches typically combine dense visual e…
DynSess: Dynamic Session-Level Evaluation and Optimization Framework for Role-Playing Agents
Rongsheng Zhang, Jiji Tang, Junnan Ren +6
Role-playing with large language models is fundamentally a session-level task, requiring agents to sustain character identity and interaction quality across extended multi-turn con…
Comprehensive Benchmarking of Long-Form Speech Generation in Diverse Scenarios
Changhao Pan, Rui Yang, Han Wang +12
Recent advances in speech generation have enabled high-fidelity synthesis, yet systematic evaluation of models under long-context conditions remains largely underexplored. A compre…
TMD-Bench: A Multi-Level Evaluation Paradigm for Music-Dance Co-Generation
Xiaoda Yang, Majun Zhang, Changhao Pan +10
Unified audio-visual generation is rapidly gaining industrial and creative relevance, enabling applications in virtual production and interactive media. However, when moving from g…
ImVideoEdit: Image-learning Video Editing via 2D Spatial Difference Attention Blocks
Jiayang Xu, Fan Zhuo, Majun Zhang +6
Current video editing models often rely on expensive paired video data, which limits their practical scalability. In essence, most video editing tasks can be formulated as a decoup…
SDiaReward: Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness
Jingyu Lu, Yuhan Wang, Fan Zhuo +7
The rapid evolution of end-to-end spoken dialogue systems demands transcending mere textual semantics to incorporate paralinguistic nuances and the spontaneous nature of human conv…
Synthetic Singers: A Review of Deep-Learning-based Singing Voice Synthesis Approaches
Changhao Pan, Dongyu Yao, Yu Zhang +4
Recent advances in singing voice synthesis (SVS) have attracted substantial attention from both academia and industry. With the advent of large language models and novel generative…
MRSAudio: A Large-Scale Multimodal Recorded Spatial Audio Dataset with Refined Annotations
Wenxiang Guo, Changhao Pan, Zhiyuan Zhu +16
Humans rely on multisensory integration to perceive spatial environments, where auditory cues enable sound source localization in three-dimensional space. Despite the critical role…
ASAudio: A Survey of Advanced Spatial Audio Research
Zhiyuan Zhu, Yu Zhang, Wenxiang Guo +2
With the rapid development of spatial audio technologies today, applications in AR, VR, and other scenarios have garnered extensive attention. Unlike traditional mono sound, spatia…
STARS: A Unified Framework for Singing Transcription, Alignment, and Refined Style Annotation
Wenxiang Guo, Yu Zhang, Changhao Pan +6
Recent breakthroughs in singing voice synthesis (SVS) have heightened the demand for high-quality annotated datasets, yet manual annotation remains prohibitively labor-intensive an…
TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis
Yu Zhang, Wenxiang Guo, Changhao Pan +6
Customizable multilingual zero-shot singing voice synthesis (SVS) has various potential applications in music composition and short video dubbing. However, existing SVS models over…
ISDrama: Immersive Spatial Drama Generation through Multimodal Prompting
Yu Zhang, Wenxiang Guo, Changhao Pan +3
Multimodal immersive spatial drama generation focuses on creating continuous multi-speaker binaural speech with dramatic prosody based on multimodal prompts, with potential applica…