91 citations · 518 across the 45 of their papers we have counts for
13 papers · 1 filter
SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation
Xutai Ma, Juan Pino, Philipp Koehn
Simultaneous text translation and end-to-end speech translation have recently made great progress but little work has combined these tasks together. We investigate how to adapt sim…
Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translation
Hang Le, Juan Pino, Changhan Wang +3
We introduce dual-decoder Transformer, a new model architecture that jointly performs automatic speech recognition (ASR) and multilingual speech translation (ST). Our models are ba…
Streaming Simultaneous Speech Translation with Augmented Memory Transformer
Xutai Ma, Yongqiang Wang, Mohammad Javad Dousti +2
Transformer-based models have achieved state-of-the-art performance on speech translation tasks. However, the model architecture is not efficient enough for streaming scenarios sin…
Multilingual Speech Translation with Efficient Finetuning of Pretrained Models
Xian Li, Changhan Wang, Yun Tang +6
We present a simple yet effective approach to build multilingual speech-to-text (ST) translation by efficient transfer learning from pretrained speech encoder and text decoder. Our…
A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks
Yun Tang, Juan Pino, Changhan Wang +2
Attention-based sequence-to-sequence modeling provides a powerful and elegant solution for applications that need to map one sequence to a different sequence. Its success heavily r…
fairseq S2T: Fast Speech-to-Text Modeling with fairseq
Changhan Wang, Yun Tang, Xutai Ma +4
We introduce fairseq S2T, a fairseq extension for speech-to-text (S2T) modeling tasks such as end-to-end speech recognition and speech-to-text translation. It follows fairseq's car…
SimulEval: An Evaluation Toolkit for Simultaneous Translation
Xutai Ma, Mohammad Javad Dousti, Changhan Wang +2
Simultaneous translation on both text and speech focuses on a real-time and low-latency scenario where the model starts translating before reading the complete source input. Evalua…
CoVoST 2 and Massively Multilingual Speech-to-Text Translation
Changhan Wang, Anne Wu, Juan Pino
Speech translation has recently become an increasingly popular topic of research, partly due to the development of benchmark datasets. Nevertheless, current datasets cover a limite…
Self-Supervised Representations Improve End-to-End Speech Translation
Anne Wu, Changhan Wang, Juan Pino +1
End-to-end speech-to-text translation can provide a simpler and smaller system but is facing the challenge of data scarcity. Pre-training methods can leverage unlabeled data and ha…
Improving Cross-Lingual Transfer Learning for End-to-End Speech Recognition with Speech Translation
Changhan Wang, Juan Pino, Jiatao Gu
Transfer learning from high-resource languages is known to be an efficient way to improve end-to-end automatic speech recognition (ASR) for low-resource languages. Pre-trained or j…
Self-Training for End-to-End Speech Translation
Juan Pino, Qiantong Xu, Xutai Ma +2
One of the main challenges for end-to-end speech translation is data scarcity. We leverage pseudo-labels generated from unlabeled audio by a cascade and an end-to-end speech transl…
SkinAugment: Auto-Encoding Speaker Conversions for Automatic Speech Translation
Arya D. McCarthy, Liezl Puzon, Juan Pino
We propose autoencoding speaker conversion for training data augmentation in automatic speech translation. This technique directly transforms an audio sequence, resulting in audio…
CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus
Changhan Wang, Juan Pino, Anne Wu +1
Spoken language translation has recently witnessed a resurgence in popularity, thanks to the development of end-to-end models and the creation of new corpora, such as Augmented Lib…