55 citations · 156 across the 9 of their papers we have counts for
4 papers · 1 filter
Reducing Activation Recomputation in Large Transformer Models
Vijay Korthikanti, Jared Casper, Sangkug Lym +4
Training large transformer models is one of the most important computational challenges of modern AI. In this paper, we show how to significantly accelerate training of large trans…
FlexSA: Flexible Systolic Array Architecture for Efficient Pruned DNN Model Training
Sangkug Lym, Mattan Erez
Modern deep learning models have high memory and computation cost. To make them fast and memory-cost efficient, structured model pruning is commonly used. We find that pruning a mo…
PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration
Sangkug Lym, Esha Choukse, Siavash Zangeneh +3
State-of-the-art convolutional neural networks (CNNs) used in vision applications have large models with numerous weights. Training these models is very compute- and memory-resourc…
Mini-batch Serialization: CNN Training with Inter-layer Data Reuse
Sangkug Lym, Armand Behroozi, Wei Wen +3
Training convolutional neural networks (CNNs) requires intense computations and high memory bandwidth. We find that bandwidth today is over-provisioned because most memory accesses…