25 citations · 110 across the 42 of their papers we have counts for
31 papers · 1 filter
Long-Prompt Zero-Shot TTS: Benchmarking Prosody Reproduction Beyond Voice Cloning
Dekun Chen, Renjie Xu, Yingda Shen +2
Zero-shot text-to-speech (TTS) typically conditions on a short speech prompt, which conveys timbre but provides only limited evidence of a speaker's prosodic patterns. We formulate…
What Matters in Preference Post-Training of Continuous Autoregressive Speech Language Models
Yingda Shen, Yao Qian, Dekun Chen +6
Continuous autoregressive speech language models combine an autoregressive (AR) backbone with a diffusion or flow-matching head. Preference post-training requires choosing how to c…
AnyEdit: A Unified Framework for Speech and Singing Voice Editing with Real-World Environmental Consistency
Yunjia Zhang, Junan Zhang, Jing Yang +3
We present AnyEdit, a unified, separation-free framework for speech and singing voice editing that preserves real-world acoustic environments, including noise, reverberation, and b…
Interpreting and Evaluating Dynamic-Rate Speech Codec Boundaries
Han Wang, Jiaqi Li, Yingda Shen +2
Dynamic-frame-rate neural speech codecs replace a uniform frame grid with variable-duration tokens, making boundary placement part of the representation itself. Yet it is unclear w…
CycleSpeech: Reciprocal Alignment for Instruction-Controlled Speech Synthesis and Paralinguistic Understanding
Huan Liao, Haonan Han, Xingwen Han +3
Instruction-controlled speech synthesis and paralinguistic understanding are often trained independently, leaving reciprocal feedback between the two tasks underexplored. We introd…
GenTraceBench: A Benchmark for Tracing Audio Deepfakes Across Pre- and Post-training Stages
Li Wang, Kunyu Feng, Wan Lin +7
Modern text-to-speech (TTS) systems are rarely deployed as unchanged pre-trained models. They are often adapted through supervised fine-tuning (SFT) or preference optimization such…
Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis
Zifan Guan, Longyu Lu, Junan Zhang +3
Evaluating live streaming speech synthesis (TTS) requires assessing fine-grained, highly expressive prosody such as emotion, intonation, and energy which traditional MOS predictors…
CoReLoop: Parameter-Efficient Controlled Recurrent Refinement for Audio Deepfake Detection
Kunyu Feng, Yuxiang Wang, Li Wang +2
Audio deepfake detectors are expected to remain robust to diverse attacks, datasets, and recording conditions, yet collecting training data that cover every future condition is imp…
SimulS2ST-Omni: Data-Efficient Streaming Speech-to-Speech Translation via Explicit Trajectory Supervision
Rongshen He, Xinyu Liang, Dekun Chen +3
Long-form streaming speech-to-speech translation (S2ST) requires incremental, unbounded translation under strict latency constraints. Existing methods typically suffer from sentenc…
Teffic-Audio: Tell Fact from Fiction
Wan Lin, Li Wang, Jindong Wang +2
Speech deepfake detection has expanded in scope with increasingly heterogeneous spoofing mechanisms, including speech synthesis, voice conversion, vocoder reconstruction, and neura…
P-MUSE: Prompt-MIDI-Optional Model for Unified Instrumental Music Synthesis and Editing
Chong Jing, Junan Zhang, Jing Yang +3
MIDI-to-Music system renders the melody and rhythm of a target MIDI sequence into musical segment while cloning instrument timbre from a prompt recording. Existing systems typicall…
Is One Score Enough? Assessing Singing Quality of Songs with Temporal Score Curves
Yishan Lv, Jing Luo, Xinyu Yang +1
Singing Quality Assessment (SQA) has become increasingly important for practical multimedia applications and Music AI systems, yet existing studies predominantly focus on short sin…
Anysynth:Zero-Shot Instrument Cloning via In-Context Learning and Asymmetric Hierarchical Guidance
Chong Jing, Junan Zhang, Jing Yang +3
Zero-shot instrument cloning aims to render an arbitrary [Target MIDI] sequence with the acoustic identity of an unseen instrument given only a short [Reference Audio, Reference MI…
Zero-VC: Zero-Lookahead Streaming Voice Conversion via Speaker Anonymization
Yudong Li, Zihao Fang, Junwen Qiu +4
Streaming zero-shot voice conversion struggles to disentangle timbre from linguistic content without degrading utility or inflating latency. Current methods rely on information bot…
FlexiSLM: A Spoken Language Model with Dynamic and Controllable Frame Rates
Jiaqi Li, Chaoren Wang, Xiaohai Tian +9
Spoken language models (SLMs) extend LLMs to speech input and output, but existing systems use fixed frame rates (e.g., 25 or 12.5 Hz), overlooking speech's time-varying informatio…
Frequency-Aware Self-Supervised Music Representation Learning
Yicheng Gu, Junan Zhang, Jerry Li +2
Self-supervised learning (SSL) has emerged as an essential paradigm for music information retrieval (MIR). While current SSL models achieve state-of-the-art performance across vari…
EigeNet: Geometry-Informed Multi-Modal Learning for Few-shot Novel View RIR Prediction
Chong Jing, Zitong Lan, Junan Zhang +1
Predicting spatially varying Room Impulse Response (RIR) from sparse observations is a critical but highly challenging inverse problem for immersive spatial audio rendering. In thi…
Aliasing-Free Neural Audio Synthesis
Yicheng Gu, Junan Zhang, Chaoren Wang +3
In neural audio synthesis, neural vocoders and codecs are models that reconstruct waveforms from acoustic and latent representations, which are essential to the resulting audio qua…
An Extensive Analysis of the Singing Voice Conversion Challenge 2025 Evaluation Results
Lester Phillip Violeta, Xueyao Zhang, Jiatong Shi +4
We present a thorough analysis of the findings of the latest iteration of the Singing Voice Conversion Challenge, a scientific event aiming to compare and understand different voic…
PicoAudio: Enabling Precise Timestamp and Frequency Controllability of Audio Events in Text-to-audio Generation
Zeyu Xie, Xuenan Xu, Zhizheng Wu +1
Recently, audio generation tasks have attracted considerable research interests. Precise temporal controllability is essential to integrate audio generation with real applications.…
AudioTime: A Temporally-aligned Audio-text Benchmark Dataset
Zeyu Xie, Xuenan Xu, Zhizheng Wu +1
Recent advancements in audio generation have enabled the creation of high-fidelity audio clips from free-form textual descriptions. However, temporal relationships, a critical feat…
An Investigation of Time-Frequency Representation Discriminators for High-Fidelity Vocoder
Yicheng Gu, Xueyao Zhang, Liumeng Xue +2
Generative Adversarial Network (GAN) based vocoders are superior in both inference speed and synthesis quality when reconstructing an audible waveform from an acoustic representati…
SingVisio: Visual Analytics of Diffusion Model for Singing Voice Conversion
Liumeng Xue, Chaoren Wang, Mingxuan Wang +3
In this study, we present SingVisio, an interactive visual analysis system that aims to explain the diffusion model used in singing voice conversion. SingVisio provides a visual di…
Amphion: An Open-Source Audio, Music and Speech Generation Toolkit
Xueyao Zhang, Liumeng Xue, Yicheng Gu +16
Amphion is an open-source toolkit for Audio, Music, and Speech Generation, targeting to ease the way for junior researchers and engineers into these fields. It presents a unified f…