Multilingual AMR Parsing with Noisy Knowledge Distillation

Abstract

We study multilingual AMR parsing from the perspective of knowledge distillation, where the aim is to learn and improve a multilingual AMR parser by using an existing English parser as its teacher. We constrain our exploration in a strict multilingual setting: there is but one model to parse all different languages including English. We identify that noisy input and precise output are the key to successful distillation. Together with extensive pre-training, we obtain an AMR parser whose performances surpass all previously published results on four different foreign languages, including German, Spanish, Italian, and Chinese, by large margins (up to 18.8 \textsc{Smatch} points on Chinese and on average 11.3 \textsc{Smatch} points). Our parser also achieves comparable performance on English to the latest state-of-the-art English-only parser.

EMNLP21 (findings)

References in corpus (5)

• Distilling the Knowledge in a Neural Network
• Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer
• Multilingual Denoising Pre-training for Neural Machine Translation
• Multilingual Translation with Extensible Multilingual Pretraining and Finetuning
• Multilingual Neural Machine Translation with Knowledge Distillation