AutoHete: An Automatic and Efficient Heterogeneous Training System for LLMs

Zihao Zeng, Chubo Liu, Xin He +5

Transformer-based large language models (LLMs) have demonstrated exceptional capabilities in sequence modeling and text generation, with improvements scaling proportionally with mo…