25 citations · 110 across the 42 of their papers we have counts for
50 papers
Repairing Speech Access After Text Updates with Frozen Affine Fields
Ming-Hao Hsu, Yuxiang Wang, Yingda Shen +1
Speech-capable language models inherit most of their knowledge from text-rich language backbones, and continued model maintenance is likewise dominated by text updates. Knowledge a…
Long-Prompt Zero-Shot TTS: Benchmarking Prosody Reproduction Beyond Voice Cloning
Dekun Chen, Renjie Xu, Yingda Shen +2
Zero-shot text-to-speech (TTS) typically conditions on a short speech prompt, which conveys timbre but provides only limited evidence of a speaker's prosodic patterns. We formulate…
What Matters in Preference Post-Training of Continuous Autoregressive Speech Language Models
Yingda Shen, Yao Qian, Dekun Chen +6
Continuous autoregressive speech language models combine an autoregressive (AR) backbone with a diffusion or flow-matching head. Preference post-training requires choosing how to c…
AnyEdit: A Unified Framework for Speech and Singing Voice Editing with Real-World Environmental Consistency
Yunjia Zhang, Junan Zhang, Jing Yang +3
We present AnyEdit, a unified, separation-free framework for speech and singing voice editing that preserves real-world acoustic environments, including noise, reverberation, and b…
Interpreting and Evaluating Dynamic-Rate Speech Codec Boundaries
Han Wang, Jiaqi Li, Yingda Shen +2
Dynamic-frame-rate neural speech codecs replace a uniform frame grid with variable-duration tokens, making boundary placement part of the representation itself. Yet it is unclear w…
CycleSpeech: Reciprocal Alignment for Instruction-Controlled Speech Synthesis and Paralinguistic Understanding
Huan Liao, Haonan Han, Xingwen Han +3
Instruction-controlled speech synthesis and paralinguistic understanding are often trained independently, leaving reciprocal feedback between the two tasks underexplored. We introd…
GenTraceBench: A Benchmark for Tracing Audio Deepfakes Across Pre- and Post-training Stages
Li Wang, Kunyu Feng, Wan Lin +7
Modern text-to-speech (TTS) systems are rarely deployed as unchanged pre-trained models. They are often adapted through supervised fine-tuning (SFT) or preference optimization such…
Multi-Dimensional Prosody Judgment For Live Streaming Speech Synthesis
Zifan Guan, Longyu Lu, Junan Zhang +3
Evaluating live streaming speech synthesis (TTS) requires assessing fine-grained, highly expressive prosody such as emotion, intonation, and energy which traditional MOS predictors…
CoReLoop: Parameter-Efficient Controlled Recurrent Refinement for Audio Deepfake Detection
Kunyu Feng, Yuxiang Wang, Li Wang +2
Audio deepfake detectors are expected to remain robust to diverse attacks, datasets, and recording conditions, yet collecting training data that cover every future condition is imp…
RecurTrace: Adaptive Latent Reasoning with Loop-Time Memory
Yuxiang Wang, Kunyu Feng, Yingda Shen +3
Repeating a small block of middle layers increases a language model's effective inference depth without adding parameters or generating extra tokens, and recent work shows that thi…
SimulS2ST-Omni: Data-Efficient Streaming Speech-to-Speech Translation via Explicit Trajectory Supervision
Rongshen He, Xinyu Liang, Dekun Chen +3
Long-form streaming speech-to-speech translation (S2ST) requires incremental, unbounded translation under strict latency constraints. Existing methods typically suffer from sentenc…
Teffic-Audio: Tell Fact from Fiction
Wan Lin, Li Wang, Jindong Wang +2
Speech deepfake detection has expanded in scope with increasingly heterogeneous spoofing mechanisms, including speech synthesis, voice conversion, vocoder reconstruction, and neura…
P-MUSE: Prompt-MIDI-Optional Model for Unified Instrumental Music Synthesis and Editing
Chong Jing, Junan Zhang, Jing Yang +3
MIDI-to-Music system renders the melody and rhythm of a target MIDI sequence into musical segment while cloning instrument timbre from a prompt recording. Existing systems typicall…
Is One Score Enough? Assessing Singing Quality of Songs with Temporal Score Curves
Yishan Lv, Jing Luo, Xinyu Yang +1
Singing Quality Assessment (SQA) has become increasingly important for practical multimedia applications and Music AI systems, yet existing studies predominantly focus on short sin…
Anysynth:Zero-Shot Instrument Cloning via In-Context Learning and Asymmetric Hierarchical Guidance
Chong Jing, Junan Zhang, Jing Yang +3
Zero-shot instrument cloning aims to render an arbitrary [Target MIDI] sequence with the acoustic identity of an unseen instrument given only a short [Reference Audio, Reference MI…
CHILDES-Aligned: A Curated Children's Speech Dataset via Multi-Model Timestamp Ensembling
Haolong Zheng, Yuanzhuo Hu, Xinyu Liang +7
CHILDES is a large-scale child speech corpus containing long-form recordings of naturalistic child-adult interactions, making it a valuable resource for studying child speech and l…
Zero-VC: Zero-Lookahead Streaming Voice Conversion via Speaker Anonymization
Yudong Li, Zihao Fang, Junwen Qiu +4
Streaming zero-shot voice conversion struggles to disentangle timbre from linguistic content without degrading utility or inflating latency. Current methods rely on information bot…
Entity Binding Failures in Speech LLM Reasoning: Diagnosis and Chain-of-Thought Intervention
Ming-Hao Hsu, Xiaohai Tian, Jun Zhang +1
Speech Large Language Models (SLLMs) underperform their text counterparts on complex reasoning. We reveal that this gap is not a uniform cognitive deficit. Evaluating two architect…
Is Text All You Need? Text as a Universal Information Bottleneck for Speech LLMs
Ming-Hao Hsu, Yuxuan Hu, Shujie Liu +3
Large language models (LLMs) provide a powerful reasoning backbone for speech understanding, but integrating continuous acoustic signals into a pretrained LLM remains challenging.…
FlexiSLM: A Spoken Language Model with Dynamic and Controllable Frame Rates
Jiaqi Li, Chaoren Wang, Xiaohai Tian +9
Spoken language models (SLMs) extend LLMs to speech input and output, but existing systems use fixed frame rates (e.g., 25 or 12.5 Hz), overlooking speech's time-varying informatio…
Frequency-Aware Self-Supervised Music Representation Learning
Yicheng Gu, Junan Zhang, Jerry Li +2
Self-supervised learning (SSL) has emerged as an essential paradigm for music information retrieval (MIR). While current SSL models achieve state-of-the-art performance across vari…
ParaBridge: Bridging Paralinguistic Perception and Dialogue Behavior in Speech Language Models
Yuxiang Wang, Qinke Ni, Shengbo Cai +3
Speech carries more information than just words: a child's voice, a fearful tone, or a noisy background should all lead a sufficiently competent spoken-dialogue assistant to differ…
EigeNet: Geometry-Informed Multi-Modal Learning for Few-shot Novel View RIR Prediction
Chong Jing, Zitong Lan, Junan Zhang +1
Predicting spatially varying Room Impulse Response (RIR) from sparse observations is a critical but highly challenging inverse problem for immersive spatial audio rendering. In thi…
VoxPrivacy: A Benchmark for Evaluating Interactional Privacy of Speech Language Models
Yuxiang Wang, Hongyu Liu, Dekun Chen +2
As Speech Language Models (SLMs) transition from personal devices to shared, multi-user environments such as smart homes, a new challenge emerges: the model is expected to distingu…