VoiceFixer: A Unified Framework for High-Fidelity Speech Restoration
arXiv:2204.05841 · doi:10.21437/Interspeech.2022-11026
Abstract
Speech restoration aims to remove distortions in speech signals. Prior methods mainly focus on a single type of distortion, such as speech denoising or dereverberation. However, speech signals can be degraded by several different distortions simultaneously in the real world. It is thus important to extend speech restoration models to deal with multiple distortions. In this paper, we introduce VoiceFixer, a unified framework for high-fidelity speech restoration. VoiceFixer restores speech from multiple distortions (e.g., noise, reverberation, and clipping) and can expand degraded speech (e.g., noisy speech) with a low bandwidth to 44.1 kHz full-bandwidth high-fidelity speech. We design VoiceFixer based on (1) an analysis stage that predicts intermediate-level features from the degraded speech, and (2) a synthesis stage that generates waveform using a neural vocoder. Both objective and subjective evaluations show that VoiceFixer is effective on severely degraded speech, such as real-world historical speech recordings. Samples of VoiceFixer are available at https://haoheliu.github.io/voicefixer.
Submitted to INTERSPEECH 2022
References in corpus (6)
- MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
- Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation
- On Filter Generalization for Music Bandwidth Extension Using Deep Neural Networks
- TFGAN: Time and Frequency Domain Based Generative Adversarial Network for High-fidelity Speech Synthesis
- CWS-PResUNet: Music Source Separation with Channel-wise Subband Phase-aware ResUNet
- Joint Echo Cancellation and Noise Suppression based on Cascaded Magnitude and Complex Mask Estimation
Cited by in corpus (6)
- HiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancement
- AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancement
- FLowHigh: Towards Efficient and High-Quality Audio Super-Resolution with Single-Step Flow Matching
- An empirical study on speech restoration guided by self supervised speech representation
- A Dual-Branch Parallel Network for Speech Enhancement and Restoration
- How much to Dereverberate? Low-Latency Single-Channel Speech Enhancement in Distant Microphone Scenarios