7 citations · 15 across the 5 of their papers we have counts for
5 papers
A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Design
Linhui Xiao, Guiping Cao, Mingyue Guo +6
The rapid expansion of large-scale AI models has led to significant performance breakthroughs across diverse domains, yet it has also raised critical concerns regarding computation…
Power Stabilization for AI Training Datacenters
Esha Choukse, Brijesh Warrier, Scot Heath +54
Large Artificial Intelligence (AI) training workloads spanning several tens of thousands of GPUs present unique power management challenges. These arise due to the high variability…
Seesaw: High-throughput LLM Inference via Model Re-sharding
Qidong Su, Wei Zhao, Xin Li +6
To improve the efficiency of distributed large language model (LLM) inference, various parallelization strategies, such as tensor and pipeline parallelism, have been proposed. Howe…
Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Models
Shang Wang, Peiming Yang, Yuxuan Zheng +2
Driven by the tremendous effort in researching novel deep learning (DL) algorithms, the training cost of developing new models increases staggeringly in recent years. We analyze GP…
Multi-node Bert-pretraining: Cost-efficient Approach
Jiahuang Lin, Xin Li, Gennady Pekhimenko
Recently, large scale Transformer-based language models such as BERT, GPT-2, and XLNet have brought about exciting leaps in state-of-the-art results for many Natural Language Proce…