241 citations · 1.2k across the 60 of their papers we have counts for
12 papers · 1 filter
Textless Speech-to-Speech Translation on Real Data
Ann Lee, Hongyu Gong, Paul-Ambroise Duquenne +8
We present a textless speech-to-speech translation (S2ST) system that can translate speech from one language into another language and can be built without the need of any text dat…
Textless Speech Emotion Conversion using Discrete and Decomposed Representations
Felix Kreuk, Adam Polyak, Jade Copet +7
Speech emotion conversion is the task of modifying the perceived emotion of a speech utterance while preserving the lexical content and speaker identity. In this study, we cast the…
Direct Simultaneous Speech-to-Speech Translation with Variational Monotonic Multihead Attention
Xutai Ma, Hongyu Gong, Danni Liu +6
We present a direct simultaneous speech-to-speech translation (Simul-S2ST) model, Furthermore, the generation of translation is independent from intermediate text representations.…
fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit
Changhan Wang, Wei-Ning Hsu, Yossi Adi +5
This paper presents fairseq S^2, a fairseq extension for speech synthesis. We implement a number of autoregressive (AR) and non-AR text-to-speech models, and their multi-speaker va…
Text-Free Prosody-Aware Generative Spoken Language Modeling
Eugene Kharitonov, Ann Lee, Adam Polyak +8
Speech pre-training has primarily demonstrated efficacy on classification tasks, while its capability of generating novel speech, similar to how GPT-2 can generate coherent paragra…
Direct speech-to-speech translation with discrete units
Ann Lee, Peng-Jen Chen, Changhan Wang +9
We present a direct speech-to-speech translation (S2ST) model that translates speech from one language to speech in another language without relying on intermediate text generation…
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai +3
Self-supervised approaches for speech representation learning are challenged by three unique problems: (1) there are multiple sound units in each input utterance, (2) there is no l…
Kaizen: Continuously improving teacher using Exponential Moving Average for semi-supervised speech recognition
Vimal Manohar, Tatiana Likhomanenko, Qiantong Xu +5
In this paper, we introduce the Kaizen framework that uses a continuously improving teacher to generate pseudo-labels for semi-supervised speech recognition (ASR). The proposed app…
Unsupervised Speech Recognition
Alexei Baevski, Wei-Ning Hsu, Alexis Conneau +1
Despite rapid progress in the recent past, current speech recognition systems still require labeled training data which limits this technology to a small fraction of the languages…
Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training
Wei-Ning Hsu, Anuroop Sriram, Alexei Baevski +8
Self-supervised learning of speech representations has been a very active research area but most work is focused on a single domain such as read audio books for which there exist l…
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations
Adam Polyak, Yossi Adi, Jade Copet +5
We propose using self-supervised discrete representations for the task of speech resynthesis. To generate disentangled representation, we separately extract low-bitrate representat…
Generative Spoken Language Modeling from Raw Audio
Kushal Lakhotia, Evgeny Kharitonov, Wei-Ning Hsu +8
We introduce Generative Spoken Language Modeling, the task of learning the acoustic and linguistic characteristics of a language from raw audio (no text, no labels), and a set of m…