180 citations · 520 across the 35 of their papers we have counts for
10 papers · 1 filter
Debunking the CUDA Myth Towards GPU-based AI Systems
Yunjae Lee, Juntaek Lim, Jehyeon Bang +10
This paper presents a comprehensive evaluation of Intel Gaudi NPUs as an alternative to NVIDIA GPUs, which is currently the de facto standard in AI system design. First, we create…
PREBA: A Hardware/Software Co-Design for Multi-Instance GPU based AI Inference Servers
Gwangoo Yeo, Jiin Kim, Yujeong Choi +1
NVIDIA's Multi-Instance GPU (MIG) is a feature that enables system designers to reconfigure one large GPU into multiple smaller GPU slices. This work characterizes this emerging GP…
ElasticRec: A Microservice-based Model Serving Architecture Enabling Elastic Resource Scaling for Recommendation Models
Yujeong Choi, Jiin Kim, Minsoo Rhu
With the increasing popularity of recommendation systems (RecSys), the demand for compute resources in datacenters has surged. However, the model-wise resource allocation employed…
Hera: A Heterogeneity-Aware Multi-Tenant Inference Server for Personalized Recommendations
Yujeong Choi, John Kim, Minsoo Rhu
While providing low latency is a fundamental requirement in deploying recommendation services, achieving high resource utility is also crucial in cost-effectively maintaining the d…
PARIS and ELSA: An Elastic Scheduling Algorithm for Reconfigurable Multi-GPU Inference Servers
Yunseong Kim, Yujeong Choi, Minsoo Rhu
In cloud machine learning (ML) inference systems, providing low latency to end-users is of utmost importance. However, maximizing server utilization and system throughput is also c…
LazyBatching: An SLA-aware Batching System for Cloud Machine Learning Inference
Yujeong Choi, Yunseong Kim, Minsoo Rhu
In cloud ML inference systems, batching is an essential technique to increase throughput which helps optimize total-cost-of-ownership. Prior graph batching combines the individual…
Centaur: A Chiplet-based, Hybrid Sparse-Dense Accelerator for Personalized Recommendations
Ranggi Hwang, Taehun Kim, Youngeun Kwon +1
Personalized recommendations are the backbone machine learning (ML) algorithm that powers several important application domains (e.g., ads, e-commerce, etc) serviced from cloud dat…
PREMA: A Predictive Multi-task Scheduling Algorithm For Preemptible Neural Processing Units
Yujeong Choi, Minsoo Rhu
To amortize cost, cloud vendors providing DNN acceleration as a service to end-users employ consolidation and virtualization to share the underlying resources among multiple DNN se…
Beyond the Memory Wall: A Case for Memory-centric HPC System for Deep Learning
Youngeun Kwon, Minsoo Rhu
As the models and the datasets to train deep learning (DL) models scale, system architects are faced with new challenges, one of which is the memory capacity bottleneck, where the…
vDNN: Virtualized Deep Neural Networks for Scalable, Memory-Efficient Neural Network Design
Minsoo Rhu, Natalia Gimelshein, Jason Clemons +2
The most widely used machine learning frameworks require users to carefully tune their memory usage so that the deep neural network (DNN) fits into the DRAM capacity of a GPU. This…