AlphaAdam:Asynchronous Masked Optimization with Dynamic Alpha for Selective Updates

Da Chang, Yu Li, Ganzhao Yuan

In the training of large language models (LLMs), updating parameters more efficiently and stably has always been an important challenge. To achieve efficient parameter updates, exi…