The NiuTrans System for the WMT21 Efficiency Task

Abstract

This paper describes the NiuTrans system for the WMT21 translation efficiency task (http://statmt.org/wmt21/efficiency-task.html). Following last year's work, we explore various techniques to improve efficiency while maintaining translation quality. We investigate the combinations of lightweight Transformer architectures and knowledge distillation strategies. Also, we improve the translation efficiency with graph optimization, low precision, dynamic batching, and parallel pre/post-processing. Our system can translate 247,000 words per second on an NVIDIA A100, being 3$\times$ faster than last year's system. Our system is the fastest and has the lowest memory consumption on the GPU-throughput track. The code, model, and pipeline will be available at NiuTrans.NMT (https://github.com/NiuTrans/NiuTrans.NMT).

NiuTrans at the WMT21 Translation Efficiency Task

References in corpus (7)

• Distilling the Knowledge in a Neural Network
• HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
• Understanding Back-Translation at Scale
• A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation
• Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Model
• FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference
• An Efficient Transformer Decoder with Compressed Sub-layers