A Memory Efficient Randomized Subspace Optimization Method for Training Large Language Models

Yiming Chen, Yuan Zhang, Yin Liu +2

The memory challenges associated with training Large Language Models (LLMs) have become a critical concern, particularly when using the Adam optimizer. To address this issue, numer…