25 citations · 39 across the 17 of their papers we have counts for
22 papers
What Matters in Preference Post-Training of Continuous Autoregressive Speech Language Models
Yingda Shen, Yao Qian, Dekun Chen +6
Continuous autoregressive speech language models combine an autoregressive (AR) backbone with a diffusion or flow-matching head. Preference post-training requires choosing how to c…
AnyEdit: A Unified Framework for Speech and Singing Voice Editing with Real-World Environmental Consistency
Yunjia Zhang, Junan Zhang, Jing Yang +3
We present AnyEdit, a unified, separation-free framework for speech and singing voice editing that preserves real-world acoustic environments, including noise, reverberation, and b…
AuraSE: Low-Hallucination Generative Speech Enhancement via Multimodal Flow Matching and Inference Policy Optimization
Yingda Shen, Yao Qian, Yuxuan Hu +7
Generative speech enhancement models can produce cleaner and more natural-sounding speech than conventional discriminative approaches, but may hallucinate by changing speech conten…
DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learning
Tristan Wu, Daniel Chin, Junan Zhang +3
Synthesizer inversion is challenging for two main reasons: 1) Distinct parameter configurations can produce perceptually similar sounds. 2) Parameter-space losses often fail to ref…
P-MUSE: Prompt-MIDI-Optional Model for Unified Instrumental Music Synthesis and Editing
Chong Jing, Junan Zhang, Jing Yang +3
MIDI-to-Music system renders the melody and rhythm of a target MIDI sequence into musical segment while cloning instrument timbre from a prompt recording. Existing systems typicall…
Anysynth:Zero-Shot Instrument Cloning via In-Context Learning and Asymmetric Hierarchical Guidance
Chong Jing, Junan Zhang, Jing Yang +3
Zero-shot instrument cloning aims to render an arbitrary [Target MIDI] sequence with the acoustic identity of an unseen instrument given only a short [Reference Audio, Reference MI…
Frequency-Aware Self-Supervised Music Representation Learning
Yicheng Gu, Junan Zhang, Jerry Li +2
Self-supervised learning (SSL) has emerged as an essential paradigm for music information retrieval (MIR). While current SSL models achieve state-of-the-art performance across vari…
EigeNet: Geometry-Informed Multi-Modal Learning for Few-shot Novel View RIR Prediction
Chong Jing, Zitong Lan, Junan Zhang +1
Predicting spatially varying Room Impulse Response (RIR) from sparse observations is a critical but highly challenging inverse problem for immersive spatial audio rendering. In thi…
Investigating training objective for flow matching-based speech enhancement
Liusha Yang, Ziru Ge, Gui Zhang +2
Speech enhancement(SE) aims to recover clean speech from noisy recordings. Although generative approaches such as score matching and Schrodinger bridge have shown strong effectiven…
Aliasing-Free Neural Audio Synthesis
Yicheng Gu, Junan Zhang, Chaoren Wang +3
In neural audio synthesis, neural vocoders and codecs are models that reconstruct waveforms from acoustic and latent representations, which are essential to the resulting audio qua…
AnyAccomp: Generalizable Accompaniment Generation via Quantized Melodic Bottleneck
Junan Zhang, Yunjia Zhang, Xueyao Zhang +1
Singing Accompaniment Generation (SAG) is the process of generating instrumental music for a given clean vocal input. However, existing SAG techniques use source-separated vocals a…
The CCF AATC 2025 Speech Restoration Challenge: A Retrospective
Junan Zhang, Mengyao Zhu, Xin Xu +3
Real-world speech communication is rarely affected by a single type of degradation. Instead, it suffers from a complex interplay of acoustic interference, codec compression, and, i…
TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modeling
Yuancheng Wang, Dekun Chen, Xueyao Zhang +3
Speech tokenizers serve as foundational components for speech language models, yet current designs exhibit several limitations, including: 1) dependence on multi-layer residual vec…
Multi-Metric Preference Alignment for Generative Speech Restoration
Junan Zhang, Xueyao Zhang, Jing Yang +3
Recent generative models have significantly advanced speech restoration tasks, yet their training objectives often misalign with human perceptual preferences, resulting in suboptim…
Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generation
Xueyao Zhang, Junan Zhang, Yuancheng Wang +5
Controllable human voice generation, particularly for expressive domains like singing, remains a significant challenge. This paper introduces Vevo2, a unified framework for control…
SingNet: Towards a Large-Scale, Diverse, and In-the-Wild Singing Voice Dataset
Yicheng Gu, Chaoren Wang, Junan Zhang +4
The lack of a publicly-available large-scale and diverse dataset has long been a significant bottleneck for singing voice applications like Singing Voice Synthesis (SVS) and Singin…
Overview of the Amphion Toolkit (v0.2)
Jiaqi Li, Xueyao Zhang, Yuancheng Wang +9
Amphion is an open-source toolkit for Audio, Music, and Speech Generation, designed to lower the entry barrier for junior researchers and engineers in these fields. It provides a v…
Metis: A Foundation Speech Generation Model with Masked Generative Pre-training
Yuancheng Wang, Jiachen Zheng, Junan Zhang +3
We introduce Metis, a foundation model for unified speech generation. Unlike previous task-specific or multi-task models, Metis follows a pre-training and fine-tuning paradigm. It…
AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancement
Junan Zhang, Jing Yang, Zihao Fang +5
We introduce AnyEnhance, a unified generative model for voice enhancement that processes both speech and singing voices. Based on a masked generative model, AnyEnhance is capable o…
LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models
Junyan Ye, Baichuan Zhou, Zilong Huang +12
With the rapid development of AI-generated content, the future internet may be inundated with synthetic data, making the discrimination of authentic and credible multimodal data in…
Leveraging Diverse Semantic-based Audio Pretrained Models for Singing Voice Conversion
Xueyao Zhang, Zihao Fang, Yicheng Gu +5
Singing Voice Conversion (SVC) is a technique that enables any singer to perform any song. To achieve this, it is essential to obtain speaker-agnostic representations from the sour…
Killing Two Birds with One Stone: Malicious Package Detection in NPM and PyPI using a Single Model of Malicious Behavior Sequence
Junan Zhang, Kaifeng Huang, Yiheng Huang +4
Open-source software (OSS) supply chain enlarges the attack surface, which makes package registries attractive targets for attacks. Recently, package registries NPM and PyPI have b…