16 papers
Enhancing Neural Speech Coding with Semantic and Visual Cues
Yao Guo, Yang Ai, Hui-Peng Du +3
At low bitrates, neural speech codecs have limited capacity to encode all information needed for high-quality re construction, especially when relying solely on speech-derived repr…
Deep Neural Compression for RIR-Characterized Acoustic Environments with Structure-Aware Constraints
Chen-Yuan Ning, Yang Ai, Hui-Peng Du +2
Room impulse responses (RIRs) characterize the acoustic environment of a room by capturing how sound propagates and decays within an enclosed space. In applications such as immersi…
PACodec: A Low-bitrate Neural Speech Codec with Parallel Additive Vector Quantization
Fei Liu, Yang Ai, Xiao-Hang Jiang +1
This paper proposes PACodec, a novel low-bitrate neural speech codec based on parallel additive vector quantization (PAVQ). Unlike the mainstream residual vector quantization (RVQ)…
Beyond WER: A Paired Acoustic Stress Test for Ambient Clinical Scribes
Xiao-Hang Jiang, Han-Jie Guo, Ying-Si Liang +4
Ambient clinical scribes increasingly combine Automatic Speech Recognition with Large Language Models to automate documentation. However, traditional metrics like Word Error Rate m…
VoCodec: A Low-bitrate Streamable Neural Speech Codec with Voicing-driven Quantization
Xiao-Hang Jiang, Yang Ai, Rui-Chen Zheng +3
Neural speech codecs are key to speech transmission and storage, but most use uniform quantization across frames, allocating the same bitrate regardless of content and wasting bits…
An Ultra-Low-Bitrate Neural Speech Codec with Plain-to-Pseudo Synergistic Vector Quantization
Xiao-Hang Jiang, Yang Ai, Fei Liu +4
Most neural speech codecs use residual vector quantization (RVQ), in which later VQs contribute less but consume the same bitrate, leading to inefficiency. We propose P2PSynCodec,…
CFMDCTCodec: A Low-Bitrate Neural Speech Codec with Noise-Prior-aware Conditional Flow Matching for MDCT-Spectral Enhancement
Xiao-Hang Jiang, Yang Ai, Hui-Peng Du +2
High-quality speech coding at low bitrates is crucial for bandwidth-constrained applications, yet remains challenging due to the severe loss of quality-critical information in high…
Ultra-Low-Bitrate Mel-Spectrogram-based Neural Speech Coding with Flow-Matching-based Refinement and Vocoding-driven Reconstruction
Hui-Peng Du, Yang Ai, Xiao-Hang Jiang +2
Ultra-low-bitrate speech coding is pivotal for bandwidth-constrained communication and deep compression, yet maintaining naturalness and speaker identity at such extreme bit budget…
CodeSep: Low-Bitrate Codec-Driven Speech Separation with Base-Token Disentanglement and Auxiliary-Token Serial Prediction
Hui-Peng Du, Yang Ai, Xiao-Hang Jiang +2
This paper targets a new scenario that integrates speech separation with speech compression, aiming to disentangle multiple speakers while producing discrete representations for ef…
A High-Quality and Low-Complexity Streamable Neural Speech Codec with Knowledge Distillation
En-Wei Zhang, Hui-Peng Du, Xiao-Hang Jiang +2
While many current neural speech codecs achieve impressive reconstructed speech quality, they often neglect latency and complexity considerations, limiting their practical deployme…
Vision-Integrated High-Quality Neural Speech Coding
Yao Guo, Yang Ai, Rui-Chen Zheng +3
This paper proposes a novel vision-integrated neural speech codec (VNSC), which aims to enhance speech coding quality by leveraging visual modality information. In VNSC, the image…
A Streamable Neural Audio Codec with Residual Scalar-Vector Quantization for Real-Time Communication
Xiao-Hang Jiang, Yang Ai, Rui-Chen Zheng +1
This paper proposes StreamCodec, a streamable neural audio codec designed for real-time communication. StreamCodec adopts a fully causal, symmetric encoder-decoder structure and op…
ESTVocoder: An Excitation-Spectral-Transformed Neural Vocoder Conditioned on Mel Spectrogram
Xiao-Hang Jiang, Hui-Peng Du, Yang Ai +2
This paper proposes ESTVocoder, a novel excitation-spectral-transformed neural vocoder within the framework of source-filter theory. The ESTVocoder transforms the amplitude and pha…
MDCTCodec: A Lightweight MDCT-based Neural Audio Codec towards High Sampling Rate and Low Bitrate Scenarios
Xiao-Hang Jiang, Yang Ai, Rui-Chen Zheng +3
In this paper, we propose MDCTCodec, an efficient lightweight end-to-end neural audio codec based on the modified discrete cosine transform (MDCT). The encoder takes the MDCT spect…
ERVQ: Enhanced Residual Vector Quantization with Intra-and-Inter-Codebook Optimization for Neural Audio Codecs
Rui-Chen Zheng, Hui-Peng Du, Xiao-Hang Jiang +2
Current neural audio codecs typically use residual vector quantization (RVQ) to discretize speech signals. However, they often experience codebook collapse, which reduces the effec…
APCodec: A Neural Audio Codec with Parallel Amplitude and Phase Spectrum Encoding and Decoding
Yang Ai, Xiao-Hang Jiang, Ye-Xin Lu +2
This paper introduces a novel neural audio codec targeting high waveform sampling rates and low bitrates named APCodec, which seamlessly integrates the strengths of parametric code…