542 citations · 1.5k across the 44 of their papers we have counts for
18 papers · 1 filter
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining
Haohe Liu, Yi Yuan, Xubo Liu +7
Although audio generation shares commonalities across different types of audio, such as speech, music, and sound effects, designing models for each type requires careful considerat…
Efficient Neural Music Generation
Max W. Y. Lam, Qiao Tian, Tang Li +10
Recent progress in music generation has been remarkably advanced by the state-of-the-art MusicLM, which comprises a hierarchy of three LMs, respectively, for semantic, coarse acous…
Joint Multi-scale Cross-lingual Speaking Style Transfer with Bidirectional Attention Mechanism for Automatic Dubbing
Jingbei Li, Sipan Li, Ping Chen +7
Automatic dubbing, which generates a corresponding version of the input speech in another language, could be widely utilized in many real-world scenarios such as video and game loc…
Neural Sound Field Decomposition with Super-resolution of Sound Direction
Qiuqiang Kong, Shilei Liu, Junjie Shi +5
Sound field decomposition predicts waveforms in arbitrary directions using signals from a limited number of microphones as inputs. Sound field decomposition is fundamental to downs…
VoiceFixer: Toward General Speech Restoration with Neural Vocoder
Haohe Liu, Qiuqiang Kong, Qiao Tian +4
Speech restoration aims to remove distortions in speech signals. Prior methods mainly focus on single-task speech restoration (SSR), such as speech denoising or speech declipping.…
Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation
Qiuqiang Kong, Yin Cao, Haohe Liu +2
Deep neural network based methods have been successfully applied to music source separation. They typically learn a mapping from a mixture spectrogram to a set of source spectrogra…
The ByteDance Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2021
Keke Wang, Xudong Mao, Hao Wu +4
This paper describes the ByteDance speaker diarization system for the fourth track of the VoxCeleb Speaker Recognition Challenge 2021 (VoxSRC-21). The VoxSRC-21 provides both the d…
Joint Echo Cancellation and Noise Suppression based on Cascaded Magnitude and Complex Mask Estimation
Xiaofeng Shu, Yehang Zhu, Yanjie Chen +4
Acoustic echo and background noise can seriously degrade the intelligibility of speech. In practice, echo and noise suppression are usually treated as two separated tasks and can b…
Audiovisual Singing Voice Separation
Bochen Li, Yuxuan Wang, Zhiyao Duan
Separating a song into vocal and accompaniment components is an active research topic, and recent years witnessed an increased performance from supervised training using deep learn…
Modeling the Compatibility of Stem Tracks to Generate Music Mashups
Jiawen Huang, Ju-Chiang Wang, Jordan B. L. Smith +2
A music mashup combines audio elements from two or more songs to create a new work. To reduce the time and effort required to make them, researchers have developed algorithms that…
Listen, Read, and Identify: Multimodal Singing Language Identification of Music
Keunwoo Choi, Yuxuan Wang
We propose a multimodal singing language classification model that uses both audio content and textual metadata. LRID-Net, the proposed model, takes an audio signal and a language…
Speech enhancement with weakly labelled data from AudioSet
Qiuqiang Kong, Haohe Liu, Xingjian Du +3
Speech enhancement is a task to improve the intelligibility and perceptual quality of degraded speech signal. Recently, neural networks based methods have been applied to speech en…
CatNet: music source separation system with mix-audio augmentation
Xuchen Song, Qiuqiang Kong, Xingjian Du +1
Music source separation (MSS) is the task of separating a music piece into individual sources, such as vocals and accompaniment. Recently, neural network based methods have been ap…
Large-Scale MIDI-based Composer Classification
Qiuqiang Kong, Keunwoo Choi, Yuxuan Wang
Music classification is a task to classify a music piece into labels such as genres or composers. We propose large-scale MIDI based composer classification systems using GiantMIDI-…
High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Times
Qiuqiang Kong, Bochen Li, Xuchen Song +2
Automatic music transcription (AMT) is the task of transcribing audio recordings into symbolic representations. Recently, neural network-based methods have been applied to AMT, and…
Adversarial Feature Learning and Unsupervised Clustering based Speech Synthesis for Found Data with Acoustic and Textual Noise
Shan Yang, Yuxuan Wang, Lei Xie
Attention-based sequence-to-sequence (seq2seq) speech synthesis has achieved extraordinary performance. But a studio-quality corpus with manual transcription is necessary to train…
Source separation with weakly labelled data: An approach to computational auditory scene analysis
Qiuqiang Kong, Yuxuan Wang, Xuchen Song +3
Source separation is the task to separate an audio recording into individual sound sources. Source separation is fundamental for computational auditory scene analysis. Previous wor…
PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition
Qiuqiang Kong, Yin Cao, Turab Iqbal +3
Audio pattern recognition is an important research topic in the machine learning area, and includes several tasks such as audio tagging, acoustic scene classification, music classi…