43 citations · 87 across the 30 of their papers we have counts for
34 papers
SURE-EVAL: A Systematic and Unified Agentic Framework for Reproducible Evaluation
Jing Peng, Junhao Du, Yixuan Wang +21
Audio and speech models are released rapidly, but reported scores often conflate model capability with deployment and evaluation choices. The same checkpoint can produce different…
Read What You Hear: Reference-Free Hypotheses Evaluation with Acoustic Discrepancy
Zhihan Li, Hankun Wang, Yiwei Guo +3
Automatic speech recognition systems commonly rely on reference transcriptions for evaluation, while reference-free approaches often depend on internal confidence estimation or aux…
HoliTok: A Continuous Holistic Tokenization with Robust Dual Capabilities of Speech Generation and Understanding
Bohan Li, Shi Lian, Hankun Wang +6
Unified speech foundation models require a holistic tokenization space that is both learnable by language models and decodable into high-quality waveforms. Existing speech tokenize…
RAS: a Reliability Oriented Metric for Automatic Speech Recognition
Wenbin Huang, Yuhang Qiu, Bohan Li +5
Automatic speech recognition systems often produce confident yet incorrect transcriptions under noisy or ambiguous conditions, which can be misleading for both users and downstream…
ISA-Bench: Benchmarking Instruction Sensitivity for Large Audio Language Models
Bohan Li, Wenbin Huang, Yuhang Qiu +7
Large Audio Language Models (LALMs), which couple acoustic perception with large language models (LLMs) to extract and understand diverse information from audio, have attracted int…
AHAMask: Reliable Task Specification for Large Audio Language Models without Instructions
Yiwei Guo, Bohan Li, Hankun Wang +4
Although current large audio language models (LALMs) extend text large language models (LLMs) with generic acoustic understanding abilities, they usually suffer from prompt sensiti…
Robust and Efficient Autoregressive Speech Synthesis with Dynamic Chunk-wise Prediction Policy
Bohan Li, Zhihan Li, Haoran Wang +5
Recently, autoregressive (AR) language models have emerged as a dominant approach in speech synthesis, offering expressive generation and scalable training. However, conventional A…
CodecSlime: Temporal Redundancy Compression of Neural Speech Codec via Dynamic Frame Rate
Hankun Wang, Yiwei Guo, Chongtian Shao +2
Neural speech codecs have been widely used in audio compression and various downstream tasks. Current mainstream codecs are fixed-frame-rate (FFR), which allocate the same number o…
Towards General Discrete Speech Codec for Complex Acoustic Environments: A Study of Reconstruction and Downstream Task Consistency
Haoran Wang, Guanyu Chen, Bohan Li +5
Neural speech codecs excel in reconstructing clean speech signals; however, their efficacy in complex acoustic environments and downstream signal processing tasks remains underexpl…
Unlocking Temporal Flexibility: Neural Speech Codec with Variable Frame Rate
Hanglei Zhang, Yiwei Guo, Zhihan Li +3
Most neural speech codecs achieve bitrate adjustment through intra-frame mechanisms, such as codebook dropout, at a Constant Frame Rate (CFR). However, speech segments inherently h…
Recent Advances in Discrete Speech Tokens: A Review
Yiwei Guo, Zhihan Li, Hankun Wang +7
The rapid advancement of speech generation technologies in the era of large language models (LLMs) has established discrete speech tokens as a foundational paradigm for speech repr…
Why Do Speech Language Models Fail to Generate Semantically Coherent Outputs? A Modality Evolving Perspective
Hankun Wang, Haoran Wang, Yiwei Guo +3
Although text-based large language models exhibit human-level writing ability and remarkable intelligence, speech language models (SLMs) still struggle to generate semantically coh…
Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decoding
Bohan Li, Hankun Wang, Situo Zhang +2
The auto-regressive architecture, like GPTs, is widely used in modern Text-to-Speech (TTS) systems. However, it incurs substantial inference time, particularly due to the challenge…
A Survey on Speech Large Language Models for Understanding
Jing Peng, Yucheng Wang, Bohan Li +9
Speech understanding is essential for interpreting the diverse forms of information embedded in spoken language, including linguistic, paralinguistic, and non-linguistic cues that…
LSCodec: Low-Bitrate and Speaker-Decoupled Discrete Speech Codec
Yiwei Guo, Zhihan Li, Chenpeng Du +3
Although discrete speech tokens have exhibited strong potential for language model-based speech generation, their high bitrates and redundant timbre information restrict the develo…
vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoders
Yiwei Guo, Zhihan Li, Junjie Li +5
We propose a new speech discrete token vocoder, vec2wav 2.0, which advances voice conversion (VC). We use discrete tokens from speech self-supervised models as the content features…
DiveSound: LLM-Assisted Automatic Taxonomy Construction for Diverse Audio Generation
Baihan Li, Zeyu Xie, Xuenan Xu +5
Audio generation has attracted significant attention. Despite remarkable enhancement in audio quality, existing models overlook diversity evaluation. This is partially due to the l…
On the Effectiveness of Acoustic BPE in Decoder-Only TTS
Bohan Li, Feiyu Shen, Yiwei Guo +3
Discretizing speech into tokens and generating them by a decoder-only model have been a promising direction for text-to-speech (TTS) and spoken language modeling (SLM). To shorten…
StoryTTS: A Highly Expressive Text-to-Speech Dataset with Rich Textual Expressiveness Annotations
Sen Liu, Yiwei Guo, Xie Chen +1
While acoustic expressiveness has long been studied in expressive text-to-speech (ETTS), the inherent expressiveness in text lacks sufficient attention, especially for ETTS of arti…
The X-LANCE Technical Report for Interspeech 2024 Speech Processing Using Discrete Speech Unit Challenge
Yiwei Guo, Chenrun Wang, Yifan Yang +9
Discrete speech tokens have been more and more popular in multiple speech processing fields, including automatic speech recognition (ASR), text-to-speech (TTS) and singing voice sy…
Attention-Constrained Inference for Robust Decoder-Only Text-to-Speech
Hankun Wang, Chenpeng Du, Yiwei Guo +3
Recent popular decoder-only text-to-speech models are known for their ability of generating natural-sounding speech. However, such models sometimes suffer from word skipping and re…
SEF-VC: Speaker Embedding Free Zero-Shot Voice Conversion with Cross Attention
Junjie Li, Yiwei Guo, Xie Chen +1
Zero-shot voice conversion (VC) aims to transfer the source speaker timbre to arbitrary unseen target speaker timbre, while keeping the linguistic content unchanged. Although the v…
VALL-T: Decoder-Only Generative Transducer for Robust and Decoding-Controllable Text-to-Speech
Chenpeng Du, Yiwei Guo, Hankun Wang +6
Recent TTS models with decoder-only Transformer architecture, such as SPEAR-TTS and VALL-E, achieve impressive naturalness and demonstrate the ability for zero-shot adaptation give…
Expressive TTS Driven by Natural Language Prompts Using Few Human Annotations
Hanglei Zhang, Yiwei Guo, Sen Liu +2
Expressive text-to-speech (TTS) aims to synthesize speeches with human-like tones, moods, or even artistic attributes. Recent advancements in expressive TTS empower users with the…