A Mean Field Ansatz for Zero-Shot Weight Transfer

Xingyuan Chen, Wenwei Kuang, Lei Deng +3

The pre-training cost of large language models (LLMs) is prohibitive. One cutting-edge approach to reduce the cost is zero-shot weight transfer, also known as model growth for some…