26 citations · 107 across the 24 of their papers we have counts for
24 papers
Generative Expressive Conversational Speech Synthesis
Rui Liu, Yifan Hu, Yi Ren +2
Conversational Speech Synthesis (CSS) aims to express a target utterance with the proper speaking style in a user-agent conversation setting. Existing CSS methods employ effective…
Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling
Rui Liu, Yifan Hu, Yi Ren +2
Conversational Speech Synthesis (CSS) aims to accurately express an utterance with the appropriate prosody and emotional inflection within a conversational setting. While recognisi…
FluentEditor: Text-based Speech Editing by Considering Acoustic and Prosody Consistency
Rui Liu, Jiatian Xi, Ziyue Jiang +1
Text-based speech editing (TSE) techniques are designed to enable users to edit the output audio by modifying the input text transcript instead of the audio itself. Despite much pr…
Emotion-Aware Prosodic Phrasing for Expressive Text-to-Speech
Rui Liu, Bin Liu, Haizhou Li
Prosodic phrasing is crucial to the naturalness and intelligibility of end-to-end Text-to-Speech (TTS). There exist both linguistic and emotional prosody in natural speech. As the…
Betray Oneself: A Novel Audio DeepFake Detection Model via Mono-to-Stereo Conversion
Rui Liu, Jinhua Zhang, Guanglai Gao +1
Audio Deepfake Detection (ADD) aims to detect the fake audio generated by text-to-speech (TTS), voice conversion (VC) and replay, etc., which is an emerging topic. Traditionally we…
MnTTS2: An Open-Source Multi-Speaker Mongolian Text-to-Speech Synthesis Dataset
Kailin Liang, Bin Liu, Yifan Hu +3
Text-to-Speech (TTS) synthesis for low-resource languages is an attractive research issue in academia and industry nowadays. Mongolian is the official language of the Inner Mongoli…
Explicit Intensity Control for Accented Text-to-speech
Rui Liu, Haolin Zuo, De Hu +2
Accented text-to-speech (TTS) synthesis seeks to generate speech with an accent (L2) as a variant of the standard version (L1). How to control the intensity of accent in the proces…
FCTalker: Fine and Coarse Grained Context Modeling for Expressive Conversational Speech Synthesis
Yifan Hu, Rui Liu, Guanglai Gao +1
Conversational Text-to-Speech (TTS) aims to synthesis an utterance with the right linguistic and affective prosody in a conversational context. The correlation between the current…
Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalities
Haolin Zuo, Rui Liu, Jinming Zhao +2
Multimodal emotion recognition leverages complementary information across modalities to gain performance. However, we cannot guarantee that the data of all modalities are always pr…
A Deep Investigation of RNN and Self-attention for the Cyrillic-Traditional Mongolian Bidirectional Conversion
Muhan Na, Rui Liu, Feilong +1
Cyrillic and Traditional Mongolian are the two main members of the Mongolian writing system. The Cyrillic-Traditional Mongolian Bidirectional Conversion (CTMBC) task includes two c…
MnTTS: An Open-Source Mongolian Text-to-Speech Synthesis Dataset and Accompanied Baseline
Yifan Hu, Pengkai Yin, Rui Liu +2
This paper introduces a high-quality open-source text-to-speech (TTS) synthesis dataset for Mongolian, a low-resource language spoken by over 10 million people worldwide. The datas…
Controllable Accented Text-to-Speech Synthesis
Rui Liu, Berrak Sisman, Guanglai Gao +1
Accented text-to-speech (TTS) synthesis seeks to generate speech with an accent (L2) as a variant of the standard version (L1). Accented TTS synthesis is challenging as L2 is diffe…
Accurate Emotion Strength Assessment for Seen and Unseen Speech Based on Data-Driven Deep Learning
Rui Liu, Berrak Sisman, Björn Schuller +2
Emotion classification of speech and assessment of the emotion strength are required in applications such as emotional text-to-speech and voice conversion. The emotion attribute ra…
StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis
Rui Liu, Berrak Sisman, Haizhou Li
Recently, emotional speech synthesis has achieved remarkable performance. The emotion strength of synthesized speech can be controlled flexibly using a strength descriptor, which i…
Decoupling Speaker-Independent Emotions for Voice Conversion Via Source-Filter Networks
Zhaojie Luo, Shoufeng Lin, Rui Liu +3
Emotional voice conversion (VC) aims to convert a neutral voice to an emotional (e.g. happy) one while retaining the linguistic information and speaker identity. We note that the d…
VisualTTS: TTS with Accurate Lip-Speech Synchronization for Automatic Voice Over
Junchen Lu, Berrak Sisman, Rui Liu +2
In this paper, we formulate a novel task to synthesize speech in sync with a silent pre-recorded video, denoted as automatic voice over (AVO). Unlike traditional speech synthesis,…
Emotional Voice Conversion: Theory, Databases and ESD
Kun Zhou, Berrak Sisman, Rui Liu +1
In this paper, we first provide a review of the state-of-the-art emotional voice conversion research, and the existing emotional speech databases. We then motivate the development…
Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability
Rui Liu, Berrak Sisman, Haizhou Li
Emotional text-to-speech synthesis (ETTS) has seen much progress in recent years. However, the generated voice is often not perceptually identifiable by its intended emotion catego…
Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset
Kun Zhou, Berrak Sisman, Rui Liu +1
Emotional voice conversion aims to transform emotional prosody in speech while preserving the linguistic content and speaker identity. Prior studies show that it is possible to dis…
GraphSpeech: Syntax-Aware Graph Attention Network For Neural Speech Synthesis
Rui Liu, Berrak Sisman, Haizhou Li
Attention-based end-to-end text-to-speech synthesis (TTS) is superior to conventional statistical methods in many ways. Transformer-based TTS is one of such successful implementati…
Modeling Prosodic Phrasing with Multi-Task Learning in Tacotron-based TTS
Rui Liu, Berrak Sisman, Feilong Bao +2
Tacotron-based end-to-end speech synthesis has shown remarkable voice quality. However, the rendering of prosody in the synthesized speech remains to be improved, especially for lo…
Expressive TTS Training with Frame and Style Reconstruction Loss
Rui Liu, Berrak Sisman, Guanglai Gao +1
We propose a novel training strategy for Tacotron-based text-to-speech (TTS) system to improve the expressiveness of speech. One of the key challenges in prosody modeling is the la…
WaveTTS: Tacotron-based TTS with Joint Time-Frequency Domain Loss
Rui Liu, Berrak Sisman, Feilong Bao +2
Tacotron-based text-to-speech (TTS) systems directly synthesize speech from text input. Such frameworks typically consist of a feature prediction network that maps character sequen…
Teacher-Student Training for Robust Tacotron-based TTS
Rui Liu, Berrak Sisman, Jingdong Li +3
While neural end-to-end text-to-speech (TTS) is superior to conventional statistical methods in many ways, the exposure bias problem in the autoregressive models remains an issue t…