AutoChunk: Automated Activation Chunk for Memory-Efficient Long Sequence Inference

Xuanlei Zhao, Shenggan Cheng, Guangyang Lu +5

Large deep learning models have achieved impressive performance across a range of applications. However, their large memory requirements, including parameter memory and activation…