LightSeq2: Accelerated Training for Transformer-based Models on GPUs

Xiaohui Wang, Yang Wei, Ying Xiong +5

Transformer-based neural models are used in many AI applications. Training these models is expensive, as it takes huge GPU resources and long duration. It is challenging because ty…