371 citations · 2.1k across the 193 of their papers we have counts for
31 papers · 1 filter
Pre-Training a Language Model Without Human Language
Cheng-Han Chiang, Hung-yi Lee
In this paper, we study how the intrinsic nature of pre-training data contributes to the fine-tuned downstream performance. To this end, we pre-train different transformer-based ma…
TaylorGAN: Neighbor-Augmented Policy Update for Sample-Efficient Natural Language Generation
Chun-Hsing Lin, Siang-Ruei Wu, Hung-Yi Lee +1
Score function-based natural language generation (NLG) approaches such as REINFORCE, in general, suffer from low sample efficiency and training instability problems. This is mainly…
How Far Are We from Robust Voice Conversion: A Survey
Tzu-hsien Huang, Jheng-hao Lin, Chien-yu Huang +1
Voice conversion technologies have been greatly improved in recent years with the help of deep learning, but their capabilities of producing natural sounding utterances in differen…
One Shot Learning for Speech Separation
Yuan-Kuei Wu, Kuan-Po Huang, Yu Tsao +1
Despite the recent success of speech separation models, they fail to separate sources properly while facing different sets of people or noisy environments. To tackle this problem,…
Hierarchical Prosody Modeling for Non-Autoregressive Speech Synthesis
Chung-Ming Chien, Hung-yi Lee
Prosody modeling is an essential component in modern text-to-speech (TTS) frameworks. By explicitly providing prosody features to the TTS model, the style of synthesized utterances…
AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalization
Yen-Hao Chen, Da-Yi Wu, Tsung-Han Wu +1
Recently, voice conversion (VC) has been widely studied. Many VC systems use disentangle-based learning techniques to separate the speaker and the linguistic content information fr…
Pretrained Language Model Embryology: The Birth of ALBERT
Cheng-Han Chiang, Sung-Feng Huang, Hung-yi Lee
While behaviors of pretrained language models (LMs) have been thoroughly examined, what happened during pretraining is rarely studied. We thus investigate the developmental process…
Semi-Supervised Spoken Language Understanding via Self-Supervised Speech and Language Model Pretraining
Cheng-I Lai, Yung-Sung Chuang, Hung-Yi Lee +2
Much recent work on Spoken Language Understanding (SLU) is limited in at least one of three ways: models were trained on oracle text input and neglected ASR errors, models were tra…
What makes multilingual BERT multilingual?
Chi-Liang Liu, Tsung-Yuan Hsu, Yung-Sung Chuang +1
Recently, multilingual BERT works remarkably well on cross-lingual transfer tasks, superior to static non-contextualized word embeddings. In this work, we provide an in-depth exper…
Stabilizing Label Assignment for Speech Separation by Self-supervised Pre-training
Sung-Feng Huang, Shun-Po Chuang, Da-Rong Liu +3
Speech separation has been well developed, with the very successful permutation invariant training (PIT) approach, although the frequent label assignment switching happening during…
FragmentVC: Any-to-Any Voice Conversion by End-to-End Extracting and Fusing Fine-Grained Voice Fragments With Attention
Yist Y. Lin, Chung-Ming Chien, Jheng-Hao Lin +2
Any-to-any voice conversion aims to convert the voice from and to any speakers even unseen during training, which is much more challenging compared to one-to-one or many-to-many ta…
Unsupervised Multiple Choices Question Answering: Start Learning from Basic Knowledge
Chi-Liang Liu, Hung-yi Lee
In this paper, we study the possibility of almost unsupervised Multiple Choices Question Answering (MCQA). Starting from very basic knowledge, MCQA model knows that some choices ha…
Looking for Clues of Language in Multilingual BERT to Improve Cross-lingual Generalization
Chi-Liang Liu, Tsung-Yuan Hsu, Yung-Sung Chuang +2
Token embeddings in multilingual BERT (m-BERT) contain both language and semantic information. We find that the representation of a language can be obtained by simply averaging the…
Investigation of Sentiment Controllable Chatbot
Hung-yi Lee, Cheng-Hao Ho, Chien-Fu Lin +5
Conventional seq2seq chatbot models attempt only to find sentences with the highest probabilities conditioned on the input sequences, without considering the sentiment of the outpu…
TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech
Andy T. Liu, Shang-Wen Li, Hung-yi Lee
We introduce a self-supervised speech pre-training method called TERA, which stands for Transformer Encoder Representations from Alteration. Recent approaches often learn by using…
VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net architecture
Da-Yi Wu, Yen-Hao Chen, Hung-Yi Lee
Voice conversion (VC) is a task that transforms the source speaker's timbre, accent, and tones in audio into another one's while preserving the linguistic content. It is still a ch…
Input-independent Attention Weights Are Expressive Enough: A Study of Attention in Self-supervised Audio Transformers
Tsung-Han Wu, Chun-Chen Hsieh, Yen-Hao Chen +2
In this paper, we seek solutions for reducing the computation complexity of transformer-based models for speech representation learning. We evaluate 10 attention algorithms; then,…
Understanding Self-Attention of Self-Supervised Audio Transformers
Shu-wen Yang, Andy T. Liu, Hung-yi Lee
Self-supervised Audio Transformers (SAT) enable great success in many downstream speech applications like ASR, but how they work has not been widely explored yet. In this work, we…
Defense for Black-box Attacks on Anti-spoofing Models by Self-Supervised Learning
Haibin Wu, Andy T. Liu, Hung-yi Lee
High-performance anti-spoofing models for automatic speaker verification (ASV), have been widely used to protect ASV by identifying and filtering spoofing audio that is deliberatel…
SADDEL: Joint Speech Separation and Denoising Model based on Multitask Learning
Yuan-Kuei Wu, Chao-I Tuan, Hung-yi Lee +1
Speech data collected in real-world scenarios often encounters two issues. First, multiple sources may exist simultaneously, and the number of sources may vary with time. Second, t…
Worse WER, but Better BLEU? Leveraging Word Embedding as Intermediate in Multitask End-to-End Speech Translation
Shun-Po Chuang, Tzu-Wei Sung, Alexander H. Liu +1
Speech translation (ST) aims to learn transformations from speech in the source language to the text in the target language. Previous works show that multitask learning improves th…
Defending Your Voice: Adversarial Attack on Voice Conversion
Chien-yu Huang, Yist Y. Lin, Hung-yi Lee +1
Substantial improvements have been achieved in recent years in voice conversion, which converts the speaker characteristics of an utterance into those of another speaker without ch…
Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio Representation
Po-Han Chi, Pei-Hung Chung, Tsung-Han Wu +4
For self-supervised speech processing, it is crucial to use pretrained models as speech representation extractors. In recent works, increasing the size of the model has been utiliz…
Semi-supervised Learning for Multi-speaker Text-to-speech Synthesis Using Discrete Speech Representation
Tao Tu, Yuan-Jui Chen, Alexander H. Liu +1
Recently, end-to-end multi-speaker text-to-speech (TTS) systems gain success in the situation where a lot of high-quality speech plus their corresponding transcriptions are availab…