514 citations · 537 across the 11 of their papers we have counts for
11 papers
MELD: Mel-Spectrogram-Based Speech Language Modeling with Discrete Latent Variables
Sung-Lin Yeh, Wei Zhou, Gil Keren +6
Recent speech language models rely on encoders that are optimized separately from autoregressive models. Since these encoders are unaware of the downstream objectives, the extracte…
Learning Speech Representations with Variational Predictive Coding
Sung-Lin Yeh, Peter Bell, Hao Tang
Despite being the best known objective for learning speech representations, the HuBERT objective has not been further developed and improved. We argue that it is the lack of an und…
Whisper Has an Internal Word Aligner
Sung-Lin Yeh, Yen Meng, Hao Tang
There is an increasing interest in obtaining accurate word-level timestamps from strong automatic speech recognizers, in particular Whisper. Existing approaches either require addi…
Estimating the Completeness of Discrete Speech Units
Sung-Lin Yeh, Hao Tang
Representing speech with discrete units has been widely used in speech codec and speech generation. However, there are several unverified claims about self-supervised discrete unit…
Open-Source Conversational AI with SpeechBrain 1.0
Mirco Ravanelli, Titouan Parcollet, Adel Moumen +30
SpeechBrain is an open-source Conversational AI toolkit based on PyTorch, focused particularly on speech processing tasks such as speech recognition, speech enhancement, speaker re…
Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective
Alexander H. Liu, Sung-Lin Yeh, James Glass
Existing studies on self-supervised speech representation learning have focused on developing new training methods and applying pre-trained models for different applications. Howev…
Learning Dependencies of Discrete Speech Representations with Neural Hidden Markov Models
Sung-Lin Yeh, Hao Tang
While discrete latent variable models have had great success in self-supervised learning, most models assume that frames are independent. Due to the segmental nature of phonemes in…
Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution
Chin-Yun Yu, Sung-Lin Yeh, György Fazekas +1
Recently, diffusion models (DMs) have been increasingly used in audio processing tasks, including speech super-resolution (SR), which aims to restore high-frequency content given l…
Autoregressive Co-Training for Learning Discrete Speech Representations
Sung-Lin Yeh, Hao Tang
While several self-supervised approaches for learning discrete speech representation have been proposed, it is unclear how these seemingly similar approaches relate to each other.…
SpeechBrain: A General-Purpose Speech Toolkit
Mirco Ravanelli, Titouan Parcollet, Peter Plantinga +18
SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to facilitate the research and development of neural speech processing technologies by being simple, fle…
Attractive or Faithful? Popularity-Reinforced Learning for Inspired Headline Generation
Yun-Zhu Song, Hong-Han Shuai, Sung-Lin Yeh +3
With the rapid proliferation of online media sources and published news, headlines have become increasingly important for attracting readers to news articles, since users may be ov…