281 citations · 1.6k across the 109 of their papers we have counts for
11 papers · 1 filter
Joint Audio and Speech Understanding
Yuan Gong, Alexander H. Liu, Hongyin Luo +2
Humans are surrounded by audio signals that include both speech and non-speech sounds. The recognition and understanding of speech and non-speech audio events, along with a profoun…
Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition
Yuan Gong, Jin Yu, James Glass
Recognizing human non-speech vocalizations is an important task and has broad applications such as automatic sound transcription and health condition monitoring. However, existing…
Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment
Yuan Gong, Ziyi Chen, Iek-Heng Chu +2
Automatic pronunciation assessment is an important technology to help self-directed language learners. While pronunciation quality has multiple aspects including accuracy, fluency,…
Simple and Effective Unsupervised Speech Synthesis
Alexander H. Liu, Cheng-I Jeff Lai, Wei-Ning Hsu +3
We introduce the first unsupervised speech synthesis system based on a simple, yet effective recipe. The framework leverages recent work in unsupervised speech recognition as well…
CMKD: CNN/Transformer-Based Cross-Model Knowledge Distillation for Audio Classification
Yuan Gong, Sameer Khurana, Andrew Rouditchenko +1
Audio classification is an active research area with a wide range of applications. Over the past decade, convolutional neural networks (CNNs) have been the de-facto standard buildi…
On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis
Cheng-I Jeff Lai, Erica Cooper, Yang Zhang +8
Are end-to-end text-to-speech (TTS) models over-parametrized? To what extent can these models be pruned, and what happens to their synthesis capabilities? This work serves as a sta…
SSAST: Self-Supervised Audio Spectrogram Transformer
Yuan Gong, Cheng-I Jeff Lai, Yu-An Chung +1
Recently, neural networks based purely on self-attention, such as the Vision Transformer (ViT), have been shown to outperform deep learning models constructed with convolutional ne…
AST: Audio Spectrogram Transformer
Yuan Gong, Yu-An Chung, James Glass
In the past decade, convolutional neural networks (CNNs) have been widely adopted as the main building block for end-to-end audio classification models, which aim to learn a direct…
PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation
Yuan Gong, Yu-An Chung, James Glass
Audio tagging is an active research area and has a wide range of applications. Since the release of AudioSet, great progress has been made in advancing model performance, which mos…
Time-Contrastive Learning Based Deep Bottleneck Features for Text-Dependent Speaker Verification
Achintya kr. Sarkar, Zheng-Hua Tan, Hao Tang +2
There are a number of studies about extraction of bottleneck (BN) features from deep neural networks (DNNs)trained to discriminate speakers, pass-phrases and triphone states for im…
Convolutional Neural Networks and Language Embeddings for End-to-End Dialect Recognition
Suwon Shon, Ahmed Ali, James Glass
Dialect identification (DID) is a special case of general language identification (LID), but a more challenging problem due to the linguistic similarity between dialects. In this p…