Acquiring Knowledge from Pre-trained Model to Neural Machine Translation

Rongxiang Weng, Heng Yu, Shujian Huang +2

Pre-training and fine-tuning have achieved great success in the natural language process field. The standard paradigm of exploiting them includes two steps: first, pre-training a m…