WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling

Jiacheng Li, Jianchao Tan, Zhidong Yang +11

Transformer architecture gradually dominates the LLM field. Recent advances in training optimization for Transformer-based large language models (LLMs) primarily focus on architect…