WATOS: Efficient LLM Training Strategies and Architecture Co-exploration for Wafer-scale Chip

Huizheng Wang, Zichuan Wang, Hongbin Wang +5

Training large language models (LLMs) imposes extreme demands on computation, memory capacity, and interconnect bandwidth, driven by their ever-increasing parameter scales and inte…