55 citations · 60 across the 2 of their papers we have counts for
6 papers
Reducing Activation Recomputation in Large Transformer Models
Vijay Korthikanti, Jared Casper, Sangkug Lym +4
Training large transformer models is one of the most important computational challenges of modern AI. In this paper, we show how to significantly accelerate training of large trans…
FlexSA: Flexible Systolic Array Architecture for Efficient Pruned DNN Model Training
Sangkug Lym, Mattan Erez
Modern deep learning models have high memory and computation cost. To make them fast and memory-cost efficient, structured model pruning is commonly used. We find that pruning a mo…
Near Data Acceleration with Concurrent Host Access
Benjamin Y. Cho, Yongkee Kwon, Sangkug Lym +1
Near-data accelerators (NDAs) that are integrated with main memory have the potential for significant power and performance benefits. Fully realizing these benefits requires the la…
DeLTA: GPU Performance Model for Deep Learning Applications with In-depth Memory System Traffic Analysis
Sangkug Lym, Donghyuk Lee, Mike O'Connor +2
Training convolutional neural networks (CNNs) requires intense compute throughput and high memory bandwidth. Especially, convolution layers account for the majority of the executio…
PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration
Sangkug Lym, Esha Choukse, Siavash Zangeneh +3
State-of-the-art convolutional neural networks (CNNs) used in vision applications have large models with numerous weights. Training these models is very compute- and memory-resourc…
Mini-batch Serialization: CNN Training with Inter-layer Data Reuse
Sangkug Lym, Armand Behroozi, Wei Wen +3
Training convolutional neural networks (CNNs) requires intense computations and high memory bandwidth. We find that bandwidth today is over-provisioned because most memory accesses…