Hecaton: Training Large Language Models with Scalable Chiplet Systems

Zongle Huang, Shupei Fan, Chen Tang +3

Large Language Models (LLMs) have achieved remarkable success in various fields, but their training and finetuning require massive computation and memory, necessitating parallelism…