371 citations · 2.1k across the 193 of their papers we have counts for
48 papers · 1 filter
SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks
Suwon Shon, Siddhant Arora, Chyi-Jiunn Lin +7
Spoken language understanding (SLU) tasks have been studied for many decades in the speech research community, but have not received as much attention as lower-level tasks like spe…
Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning
Shih-Cheng Huang, Shih-Heng Wang, Min-Han Shih +2
Parameter-efficient (PE) methods (like Prompts or Adapters) for adapting pre-trained language models (PLM) to downstream tasks have been popular recently. However, hindrances still…
Introducing Semantics into Speech Encoders
Derek Xu, Shuyan Dong, Changhan Wang +10
Recent studies find existing self-supervised speech encoders contain primarily acoustic rather than semantic information. As a result, pipelined supervised automatic speech recogni…
Bridging Speech and Textual Pre-trained Models with Unsupervised ASR
Jiatong Shi, Chan-Jan Hsu, Holam Chung +5
Spoken language understanding (SLU) is a task aiming to extract high-level semantics from spoken utterances. Previous works have investigated the use of speech self-supervised mode…
EURO: ESPnet Unsupervised ASR Open-source Toolkit
Dongji Gao, Jiatong Shi, Shun-Po Chuang +4
This paper describes the ESPnet Unsupervised ASR Open-source Toolkit (EURO), an end-to-end open-source toolkit for unsupervised automatic speech recognition (UASR). EURO adopts the…
Model Extraction Attack against Self-supervised Speech Models
Tsu-Yuan Hsu, Chen-An Li, Tung-Yu Wu +1
Self-supervised learning (SSL) speech models generate meaningful representations of given clips and achieve incredible performance across various downstream tasks. Model extraction…
Is Smaller Always Faster? Tradeoffs in Compressing Self-Supervised Speech Transformers
Tzu-Quan Lin, Tsung-Huan Yang, Chun-Yao Chang +4
Transformer-based self-supervised models have achieved remarkable success in speech processing, but their large size and high inference cost present significant challenges for real…
MelHuBERT: A simplified HuBERT on Mel spectrograms
Tzu-Quan Lin, Hung-yi Lee, Hao Tang
Self-supervised models have had great success in learning speech representations that can generalize to various downstream tasks. However, most self-supervised models require a lar…
Once-for-All Sequence Compression for Self-Supervised Speech Models
Hsuan-Jui Chen, Yen Meng, Hung-yi Lee
The sequence length along the time axis is often the dominant factor of the computation in speech processing. Works have been proposed to reduce the sequence length for lowering th…
M-SpeechCLIP: Leveraging Large-Scale, Pre-Trained Models for Multilingual Speech to Image Retrieval
Layne Berry, Yi-Jen Shih, Hsuan-Fu Wang +3
This work investigates the use of large-scale, English-only pre-trained models (CLIP and HuBERT) for multilingual image-speech retrieval. For non-English image-speech retrieval, we…
T5lephone: Bridging Speech and Text Self-supervised Models for Spoken Language Understanding via Phoneme level T5
Chan-Jan Hsu, Ho-Lam Chung, Hung-yi Lee +1
In Spoken language understanding (SLU), a natural solution is concatenating pre-trained speech models (e.g. HuBERT) and pretrained language models (PLM, e.g. T5). Most previous wor…
SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning
Tzu-hsun Feng, Annie Dong, Ching-Feng Yeh +11
We present the SUPERB challenge at SLT 2022, which aims at learning self-supervised speech representation for better performance, generalization, and efficiency. The challenge buil…
On the Utility of Self-supervised Models for Prosody-related Tasks
Guan-Ting Lin, Chi-Luen Feng, Wei-Ping Huang +5
Self-Supervised Learning (SSL) from speech data has produced models that have achieved remarkable performance in many tasks, and that are known to implicitly represent many aspects…
On Compressing Sequences for Self-Supervised Speech Models
Yen Meng, Hsuan-Jui Chen, Jiatong Shi +4
Compressing self-supervised models has become increasingly necessary, as self-supervised models become larger. While previous approaches have primarily focused on compressing the m…
SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model
Yi-Jen Shih, Hsuan-Fu Wang, Heng-Jui Chang +3
Data-driven speech processing models usually perform well with a large amount of text supervision, but collecting transcribed speech data is costly. Therefore, we propose SpeechCLI…
Improving generalizability of distilled self-supervised speech processing models under distorted settings
Kuan-Po Huang, Yu-Kuan Fu, Tsu-Yuan Hsu +5
Self-supervised learned (SSL) speech pre-trained models perform well across various speech processing tasks. Distilled versions of SSL models have been developed to match the needs…
Push-Pull: Characterizing the Adversarial Robustness for Audio-Visual Active Speaker Detection
Xuanjun Chen, Haibin Wu, Helen Meng +2
Audio-visual active speaker detection (AVASD) is well-developed, and now is an indispensable front-end for several multi-modal applications. However, to the best of our knowledge,…
Multimodal Transformer Distillation for Audio-Visual Synchronization
Xuanjun Chen, Haibin Wu, Chung-Che Wang +2
Audio-visual synchronization aims to determine whether the mouth movements and speech in the video are synchronized. VocaLiST reaches state-of-the-art performance by incorporating…
Exploring Efficient-tuning Methods in Self-supervised Speech Models
Zih-Ching Chen, Chin-Lun Fu, Chih-Ying Liu +2
In this study, we aim to explore efficient tuning methods for speech self-supervised learning. Recent studies show that self-supervised learning (SSL) can learn powerful representa…
The Efficacy of Self-Supervised Speech Models for Audio Representations
Tung-Yu Wu, Chen-An Li, Tzu-Han Lin +2
Self-supervised learning (SSL) speech models, which can serve as powerful upstream models to extract meaningful speech representations, have achieved unprecedented success in speec…
Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Network
Da-rong Liu, Po-chun Hsu, Yi-chen Chen +4
ASR has been shown to achieve great performance recently. However, most of them rely on massive paired data, which is not feasible for low-resource languages worldwide. This paper…
Meta Learning for Natural Language Processing: A Survey
Hung-yi Lee, Shang-Wen Li, Ngoc Thang Vu
Deep learning has been the mainstream technique in natural language processing (NLP) area. However, the techniques require many labeled data and are less generalizable across domai…
Tackling Spoofing-Aware Speaker Verification with Multi-Model Fusion
Haibin Wu, Jiawen Kang, Lingwei Meng +5
Recent years have witnessed the extraordinary development of automatic speaker verification (ASV). However, previous works show that state-of-the-art ASV models are seriously vulne…
Few-Shot Cross-Lingual TTS Using Transferable Phoneme Embedding
Wei-Ping Huang, Po-Chun Chen, Sung-Feng Huang +1
This paper studies a transferable phoneme embedding framework that aims to deal with the cross-lingual text-to-speech (TTS) problem under the few-shot setting. Transfer learning is…