Dynamic Stashing Quantization for Efficient Transformer Training

Guo Yang, Daniel Lo, Robert Mullins +1

Large Language Models (LLMs) have demonstrated impressive performance on a range of Natural Language Processing (NLP) tasks. Unfortunately, the immense amount of computations and m…