45 citations · 301 across the 53 of their papers we have counts for
12 papers · 1 filter
M3ST: Mix at Three Levels for Speech Translation
Xuxin Cheng, Qianqian Dong, Fengpeng Yue +3
How to solve the data scarcity problem for end-to-end speech-to-text translation (ST)? It's well known that data augmentation is an efficient method to improve performance for many…
Beyond Triplet: Leveraging the Most Data for Multimodal Machine Translation
Yaoming Zhu, Zewei Sun, Shanbo Cheng +3
Multimodal machine translation (MMT) aims to improve translation quality by incorporating information from other modalities, such as vision. Previous MMT systems mainly focus on be…
SESCORE2: Learning Text Generation Evaluation via Synthesizing Realistic Mistakes
Wenda Xu, Xian Qian, Mingxuan Wang +2
Is it possible to train a general metric for evaluating text generation quality without human annotated ratings? Existing learned metrics either perform unsatisfactorily across tex…
Leveraging per Image-Token Consistency for Vision-Language Pre-training
Yunhao Gou, Tom Ko, Hansi Yang +3
Most existing vision-language pre-training (VLP) approaches adopt cross-modal masked language modeling (CMLM) to learn vision-language associations. However, we find that CMLM is i…
PARAGEN : A Parallel Generation Toolkit
Jiangtao Feng, Yi Zhou, Jun Zhang +7
PARAGEN is a PyTorch-based NLP toolkit for further development on parallel generation. PARAGEN provides thirteen types of customizable plugins, helping users to experiment quickly…
CoBERT: Self-Supervised Speech Representation Learning Through Code Representation Learning
Chutong Meng, Junyi Ao, Tom Ko +2
Speech is the surface form of a finite set of phonetic units, which can be represented by discrete codes. We propose the Code BERT (CoBERT) approach for self-supervised speech repr…
Zero-shot Domain Adaptation for Neural Machine Translation with Retrieved Phrase-level Prompts
Zewei Sun, Qingnan Jiang, Shujian Huang +3
Domain adaptation is an important challenge for neural machine translation. However, the traditional fine-tuning solution requires multiple extra training and yields a high cost. I…
Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation
Qianqian Dong, Fengpeng Yue, Tom Ko +3
Direct Speech-to-speech translation (S2ST) has drawn more and more attention recently. The task is very challenging due to data scarcity and complex speech-to-speech mapping. In th…
Cross-modal Contrastive Learning for Speech Translation
Rong Ye, Mingxuan Wang, Lei Li
How can we learn unified representations for spoken utterances and their written text? Learning similar representations for semantically similar speech and text is important for sp…
GigaST: A 10,000-hour Pseudo Speech Translation Corpus
Rong Ye, Chengqi Zhao, Tom Ko +4
This paper introduces GigaST, a large-scale pseudo speech translation (ST) corpus. We create the corpus by translating the text in GigaSpeech, an English ASR corpus, into German an…
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation
Qingkai Fang, Rong Ye, Lei Li +2
How to learn a better speech representation for end-to-end speech-to-text translation (ST) with limited labeled data? Existing techniques often attempt to transfer powerful machine…
Unified Multimodal Punctuation Restoration Framework for Mixed-Modality Corpus
Yaoming Zhu, Liwei Wu, Shanbo Cheng +1
The punctuation restoration task aims to correctly punctuate the output transcriptions of automatic speech recognition systems. Previous punctuation models, either using text only…