371 citations · 2.1k across the 193 of their papers we have counts for
32 papers · 1 filter
Membership Inference Attacks Against Self-supervised Speech Models
Wei-Cheng Tseng, Wei-Tsung Kao, Hung-yi Lee
Recently, adapting the idea of self-supervised learning (SSL) on continuous speech has started gaining attention. SSL models pre-trained on a huge amount of unlabeled audio can gen…
Characterizing the adversarial vulnerability of speech self-supervised learning
Haibin Wu, Bo Zheng, Xu Li +3
A leaderboard named Speech processing Universal PERformance Benchmark (SUPERB), which aims at benchmarking the performance of a shared self-supervised learning (SSL) speech model a…
Meta-TTS: Meta-Learning for Few-Shot Speaker Adaptive Text-to-Speech
Sung-Feng Huang, Chyi-Jiunn Lin, Da-Rong Liu +2
Personalizing a speech synthesis system is a highly desired application, where the system can generate speech with the user's voice with rare enrolled recordings. There are two mai…
Speech Representation Learning Through Self-supervised Pretraining And Multi-task Finetuning
Yi-Chen Chen, Shu-wen Yang, Cheng-Kuang Lee +2
Speech representation learning plays a vital role in speech processing. Among them, self-supervised learning (SSL) has become an important research direction. It has been shown tha…
S3PRL-VC: Open-source Voice Conversion Framework with Self-supervised Speech Representations
Wen-Chin Huang, Shu-Wen Yang, Tomoki Hayashi +3
This paper introduces S3PRL-VC, an open-source voice conversion (VC) framework based on the S3PRL toolkit. In the context of recognition-synthesis VC, self-supervised speech repres…
An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition
Xuankai Chang, Takashi Maekaku, Pengcheng Guo +8
Self-supervised pretraining on speech data has achieved a lot of progress. High-fidelity representation of the speech signal is learned from a lot of untranscribed data and shows p…
CheerBots: Chatbots toward Empathy and Emotionusing Reinforcement Learning
Jiun-Hao Jhan, Chao-Peng Liu, Shyh-Kang Jeng +1
Apart from the coherence and fluency of responses, an empathetic chatbot emphasizes more on people's feelings. By considering altruistic behaviors between human interaction, empath…
Mandarin-English Code-switching Speech Recognition with Self-supervised Speech Representation Models
Liang-Hsuan Tseng, Yu-Kuan Fu, Heng-Jui Chang +1
Code-switching (CS) is common in daily conversations where more than one language is used within a sentence. The difficulties of CS speech recognition lie in alternating languages…
Don't speak too fast: The impact of data bias on self-supervised speech models
Yen Meng, Yi-Hui Chou, Andy T. Liu +1
Self-supervised Speech Models (S3Ms) have been proven successful in many speech downstream tasks, like ASR. However, how pre-training data affects S3Ms' downstream behavior remains…
Toward Degradation-Robust Voice Conversion
Chien-yu Huang, Kai-Wei Chang, Hung-yi Lee
Any-to-any voice conversion technologies convert the vocal timbre of an utterance to any speaker even unseen during training. Although there have been several state-of-the-art any-…
Analyzing the Robustness of Unsupervised Speech Recognition
Guan-Ting Lin, Chan-Jan Hsu, Da-Rong Liu +2
Unsupervised speech recognition (unsupervised ASR) aims to learn the ASR system with non-parallel speech and text corpus only. Wav2vec-U has shown promising results in unsupervised…
DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT
Heng-Jui Chang, Shu-wen Yang, Hung-yi Lee
Self-supervised speech representation learning methods like wav2vec 2.0 and Hidden-unit BERT (HuBERT) leverage unlabeled speech data for pre-training and offer good representations…
On the Transferability of Pre-trained Language Models: A Study from Artificial Datasets
Cheng-Han Chiang, Hung-yi Lee
Pre-training language models (LMs) on large-scale unlabeled text data makes the model much easier to achieve exceptional downstream performance than their counterparts directly tra…
Parallelized Reverse Curriculum Generation
Zih-Yun Chiu, Yi-Lin Tuan, Hung-yi Lee +1
For reinforcement learning (RL), it is challenging for an agent to master a task that requires a specific series of actions due to sparse rewards. To solve this problem, reverse cu…
Voting for the right answer: Adversarial defense for speaker verification
Haibin Wu, Yang Zhang, Zhiyong Wu +2
Automatic speaker verification (ASV) is a well developed technology for biometric identification, and has been ubiquitous implemented in security-critic applications, such as banki…
Adversarial Sample Detection for Speaker Verification by Neural Vocoders
Haibin Wu, Po-chun Hsu, Ji Gao +6
Automatic speaker verification (ASV), one of the most important technology for biometric identification, has been widely adopted in security-critical applications. However, ASV is…
Mitigating Biases in Toxic Language Detection through Invariant Rationalization
Yung-Sung Chuang, Mingye Gao, Hongyin Luo +4
Automatic detection of toxic language plays an essential role in protecting social media users, especially minority groups, from verbal abuse. However, biases toward some attribute…
Multi-accent Speech Separation with One Shot Learning
Kuan-Po Huang, Yuan-Kuei Wu, Hung-yi Lee
Speech separation is a problem in the field of speech processing that has been studied in full swing recently. However, there has not been much work studying a multi-accent speech…
Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning
Haibin Wu, Xu Li, Andy T. Liu +3
Previous works have shown that automatic speaker verification (ASV) is seriously vulnerable to malicious spoofing attacks, such as replay, synthetic speech, and recently emerged ad…
SpeechNet: A Universal Modularized Model for Speech Processing Tasks
Yi-Chen Chen, Po-Han Chi, Shu-wen Yang +7
There is a wide variety of speech processing tasks ranging from extracting content information from speech signals to generating speech signals. For different tasks, model networks…
Investigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech Translation
Shun-Po Chuang, Yung-Sung Chuang, Chih-Chiang Chang +1
We study the possibilities of building a non-autoregressive speech-to-text translation model using connectionist temporal classification (CTC), and use CTC-based automatic speech r…
Improving Cross-Lingual Reading Comprehension with Self-Training
Wei-Cheng Huang, Chien-yu Huang, Hung-yi Lee
Substantial improvements have been made in machine reading comprehension, where the machine answers questions based on a given context. Current state-of-the-art models even surpass…
SUPERB: Speech processing Universal PERformance Benchmark
Shu-wen Yang, Po-Han Chi, Yung-Sung Chuang +17
Self-supervised learning (SSL) has proven vital for advancing research in natural language processing (NLP) and computer vision (CV). The paradigm pretrains a shared model on large…
Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speech
Chung-Ming Chien, Jheng-Hao Lin, Chien-yu Huang +2
The few-shot multi-speaker multi-style voice cloning task is to synthesize utterances with voice and speaking style similar to a reference speaker given only a few reference sample…