Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understanding
arXiv:2109.01583
Abstract
Lack of training data presents a grand challenge to scaling out spoken language understanding (SLU) to low-resource languages. Although various data augmentation approaches have been proposed to synthesize training data in low-resource target languages, the augmented data sets are often noisy, and thus impede the performance of SLU models. In this paper we focus on mitigating noise in augmented data. We develop a denoising training approach. Multiple models are trained with data produced by various augmented methods. Those models provide supervision signals to each other. The experimental results show that our method outperforms the existing state of the art by 3.05 and 4.24 percentage points on two benchmark datasets, respectively. The code will be made open sourced on github.
arXiv commentLong paper at EMNLP 2021
References in corpus (9)
- Decoupled Weight Decay Regularization
- Tutorial on Variational Autoencoders
- Multilingual Denoising Pre-training for Neural Machine Translation
- Rethinking Pre-training and Self-training
- Data Augmentation Using GANs
- Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
- Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling
- Unsupervised Cross-lingual Adaptation for Sequence Tagging and Beyond
- Paraphrase Augmented Task-Oriented Dialog Generation