Demand Layering for Real-Time DNN Inference with Minimized Memory Usage

Mingoo Ji, Saehanseul Yi, Changjin Koo +4

When executing a deep neural network (DNN), its model parameters are loaded into GPU memory before execution, incurring a significant GPU memory burden. There are studies that redu…