125 citations · 462 across the 47 of their papers we have counts for
19 papers · 1 filter
DopplerBAS: Binaural Audio Synthesis Addressing Doppler Effect
Jinglin Liu, Zhenhui Ye, Qian Chen +4
Recently, binaural audio synthesis (BAS) has emerged as a promising research field for its applications in augmented and virtual realities. Binaural audio helps users orient themse…
VarietySound: Timbre-Controllable Video to Sound Generation via Unsupervised Information Disentanglement
Chenye Cui, Yi Ren, Jinglin Liu +2
Video to sound generation aims to generate realistic and natural sound given a video input. However, previous video-to-sound generation methods can only generate a random or averag…
Diffusion Denoising Process for Perceptron Bias in Out-of-distribution Detection
Luping Liu, Yi Ren, Xize Cheng +3
Out-of-distribution (OOD) detection is a crucial task for ensuring the reliability and safety of deep learning. Currently, discriminator models outperform other methods in this reg…
Video-Guided Curriculum Learning for Spoken Video Grounding
Yan Xia, Zhou Zhao, Shangwei Ye +3
In this paper, we introduce a new task, spoken video grounding (SVG), which aims to localize the desired video fragments from spoken language descriptions. Compared with using text…
AntCritic: Argument Mining for Free-Form and Visually-Rich Financial Comments
Huadai Liu, Wenqiang Xu, Xuan Lin +3
Argument mining aims to detect all possible argumentative components and identify their relationships automatically. As a thriving task in natural language processing, there has be…
ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech
Rongjie Huang, Zhou Zhao, Huadai Liu +3
Denoising diffusion probabilistic models (DDPMs) have recently achieved leading performances in many generative tasks. However, the inherited iterative sampling process costs hinde…
FastLTS: Non-Autoregressive End-to-End Unconstrained Lip-to-Speech Synthesis
Yongqi Wang, Zhou Zhao
Unconstrained lip-to-speech synthesis aims to generate corresponding speeches from silent videos of talking faces with no restriction on head poses or vocabulary. Current works mai…
AntPivot: Livestream Highlight Detection via Hierarchical Attention Mechanism
Yang Zhao, Xuan Lin, Wenqiang Xu +3
In recent days, streaming technology has greatly promoted the development in the field of livestream. Due to the excessive length of livestream records, it's quite essential to ext…
Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech
Ziyue Jiang, Zhe Su, Zhou Zhao +4
Polyphone disambiguation aims to capture accurate pronunciation knowledge from natural text sequences for reliable Text-to-speech (TTS) systems. However, previous approaches requir…
TranSpeech: Speech-to-Speech Translation With Bilateral Perturbation
Rongjie Huang, Jinglin Liu, Huadai Liu +4
Direct speech-to-speech translation (S2ST) with discrete units leverages recent progress in speech representation learning. Specifically, a sequence of discrete representations der…
GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speech
Rongjie Huang, Yi Ren, Jinglin Liu +2
Style transfer for out-of-domain (OOD) speech synthesis aims to generate speech samples with unseen style (e.g., speaker identity, emotion, and prosody) derived from an acoustic re…
SyntaSpeech: Syntax-Aware Generative Adversarial Text-to-Speech
Zhenhui Ye, Zhou Zhao, Yi Ren +1
The recent progress in non-autoregressive text-to-speech (NAR-TTS) has made fast and high-quality speech synthesis possible. However, current NAR-TTS models usually use phoneme seq…
FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis
Rongjie Huang, Max W. Y. Lam, Jun Wang +4
Denoising diffusion probabilistic models (DDPMs) have recently achieved leading performances in many generative tasks. However, the inherited iterative sampling process costs hinde…
Contrastive Learning with Positive-Negative Frame Mask for Music Representation
Dong Yao, Zhou Zhao, Shengyu Zhang +4
Self-supervised learning, especially contrastive learning, has made an outstanding contribution to the development of many deep learning research fields. Recently, researchers in t…
Learning the Beauty in Songs: Neural Singing Voice Beautifier
Jinglin Liu, Chengxi Li, Yi Ren +2
We are interested in a novel task, singing voice beautifying (SVB). Given the singing voice of an amateur singer, SVB aims to improve the intonation and vocal tone of the voice, wh…
Revisiting Over-Smoothness in Text to Speech
Yi Ren, Xu Tan, Tao Qin +2
Non-autoregressive text to speech (NAR-TTS) models have attracted much attention from both academia and industry due to their fast generation speed. One limitation of NAR-TTS model…
ProsoSpeech: Enhancing Prosody With Quantized Vector Pre-training in Text-to-Speech
Yi Ren, Ming Lei, Zhiying Huang +4
Expressive text-to-speech (TTS) has become a hot research topic recently, mainly focusing on modeling prosody in speech. Prosody modeling has several challenges: 1) the extracted p…
Pseudo Numerical Methods for Diffusion Models on Manifolds
Luping Liu, Yi Ren, Zhijie Lin +1
Denoising Diffusion Probabilistic Models (DDPMs) can generate high-quality samples such as image and audio samples. However, DDPMs require hundreds to thousands of iterations to pr…
MR-SVS: Singing Voice Synthesis with Multi-Reference Encoder
Shoutong Wang, Jinglin Liu, Yi Ren +3
Multi-speaker singing voice synthesis is to generate the singing voice sung by different speakers. To generalize to new speakers, previous zero-shot singing adaptation methods obta…