45 citations · 301 across the 53 of their papers we have counts for
47 papers · 1 filter
Speech Translation with Large Language Models: An Industrial Practice
Zhichao Huang, Rong Ye, Tom Ko +4
Given the great success of large language models (LLMs) across various tasks, in this paper, we introduce LLM-ST, a novel and effective speech translation model constructed upon a…
BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Training
Yiming Yan, Tao Wang, Chengqi Zhao +3
Automatic metrics play a crucial role in machine translation. Despite the widespread use of n-gram-based metrics, there has been a recent surge in the development of pre-trained mo…
Recent Advances in Direct Speech-to-text Translation
Chen Xu, Rong Ye, Qianqian Dong +5
Recently, speech-to-text translation has attracted more and more attention and many studies have emerged rapidly. In this paper, we present a comprehensive survey on direct speech…
PolyVoice: Language Models for Speech to Speech Translation
Qianqian Dong, Zhiying Huang, Qiao Tian +15
We propose PolyVoice, a language model-based framework for speech-to-speech translation (S2ST) system. Our framework consists of two language models: a translation language model a…
CTC-based Non-autoregressive Speech Translation
Chen Xu, Xiaoqian Liu, Xiaowen Liu +9
Combining end-to-end speech translation (ST) and non-autoregressive (NAR) generation is promising in language and speech processing for their advantages of less error propagation a…
DUB: Discrete Unit Back-translation for Speech Translation
Dong Zhang, Rong Ye, Tom Ko +2
How can speech-to-text translation (ST) perform as well as machine translation (MT)? The key point is to bridge the modality gap between speech and text so that useful MT technique…
DINOISER: Diffused Conditional Sequence Learning by Manipulating Noises
Jiasheng Ye, Zaixiang Zheng, Yu Bao +2
While diffusion models have achieved great success in generating continuous signals such as images and audio, it remains elusive for diffusion models in learning discrete sequence…
M3ST: Mix at Three Levels for Speech Translation
Xuxin Cheng, Qianqian Dong, Fengpeng Yue +3
How to solve the data scarcity problem for end-to-end speech-to-text translation (ST)? It's well known that data augmentation is an efficient method to improve performance for many…
Beyond Triplet: Leveraging the Most Data for Multimodal Machine Translation
Yaoming Zhu, Zewei Sun, Shanbo Cheng +3
Multimodal machine translation (MMT) aims to improve translation quality by incorporating information from other modalities, such as vision. Previous MMT systems mainly focus on be…
SESCORE2: Learning Text Generation Evaluation via Synthesizing Realistic Mistakes
Wenda Xu, Xian Qian, Mingxuan Wang +2
Is it possible to train a general metric for evaluating text generation quality without human annotated ratings? Existing learned metrics either perform unsatisfactorily across tex…
PARAGEN : A Parallel Generation Toolkit
Jiangtao Feng, Yi Zhou, Jun Zhang +7
PARAGEN is a PyTorch-based NLP toolkit for further development on parallel generation. PARAGEN provides thirteen types of customizable plugins, helping users to experiment quickly…
Zero-shot Domain Adaptation for Neural Machine Translation with Retrieved Phrase-level Prompts
Zewei Sun, Qingnan Jiang, Shujian Huang +3
Domain adaptation is an important challenge for neural machine translation. However, the traditional fine-tuning solution requires multiple extra training and yields a high cost. I…
Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation
Qianqian Dong, Fengpeng Yue, Tom Ko +3
Direct Speech-to-speech translation (S2ST) has drawn more and more attention recently. The task is very challenging due to data scarcity and complex speech-to-speech mapping. In th…
Cross-modal Contrastive Learning for Speech Translation
Rong Ye, Mingxuan Wang, Lei Li
How can we learn unified representations for spoken utterances and their written text? Learning similar representations for semantically similar speech and text is important for sp…
GigaST: A 10,000-hour Pseudo Speech Translation Corpus
Rong Ye, Chengqi Zhao, Tom Ko +4
This paper introduces GigaST, a large-scale pseudo speech translation (ST) corpus. We create the corpus by translating the text in GigaSpeech, an English ASR corpus, into German an…
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation
Qingkai Fang, Rong Ye, Lei Li +2
How to learn a better speech representation for end-to-end speech-to-text translation (ST) with limited labeled data? Existing techniques often attempt to transfer powerful machine…
Unified Multimodal Punctuation Restoration Framework for Mixed-Modality Corpus
Yaoming Zhu, Liwei Wu, Shanbo Cheng +1
The punctuation restoration task aims to correctly punctuate the output transcriptions of automatic speech recognition systems. Previous punctuation models, either using text only…
LightSeq2: Accelerated Training for Transformer-based Models on GPUs
Xiaohui Wang, Yang Wei, Ying Xiong +5
Transformer-based neural models are used in many AI applications. Training these models is expensive, as it takes huge GPU resources and long duration. It is challenging because ty…
The Volctrans GLAT System: Non-autoregressive Translation Meets WMT21
Lihua Qian, Yi Zhou, Zaixiang Zheng +7
This paper describes the Volctrans' submission to the WMT21 news translation shared task for German->English translation. We build a parallel (i.e., non-autoregressive) translation…
Multilingual Translation via Grafting Pre-trained Language Models
Zewei Sun, Mingxuan Wang, Lei Li
Can pre-trained BERT for one language and GPT for another be glued together to translate texts? Self-supervised training using only monolingual data has led to the success of pre-t…
Learning Kernel-Smoothed Machine Translation with Retrieved Examples
Qingnan Jiang, Mingxuan Wang, Jun Cao +3
How to effectively adapt neural machine translation (NMT) models according to emerging cases without retraining? Despite the great success of neural machine translation, updating t…
Learning When to Translate for Streaming Speech
Qianqian Dong, Yaoming Zhu, Mingxuan Wang +1
How to find proper moments to generate partial sentence translation given a streaming speech input? Existing approaches waiting-and-translating for a fixed duration often break the…
Secoco: Self-Correcting Encoding for Neural Machine Translation
Tao Wang, Chengqi Zhao, Mingxuan Wang +3
This paper presents Self-correcting Encoding (Secoco), a framework that effectively deals with input noise for robust neural machine translation by introducing self-correcting pred…
The Volctrans Neural Speech Translation System for IWSLT 2021
Chengqi Zhao, Zhicheng Liu, Jian Tong +6
This paper describes the systems submitted to IWSLT 2021 by the Volctrans team. We participate in the offline speech translation and text-to-text simultaneous translation tracks. F…