Understanding Capacity-Driven Scale-Out Neural Recommendation Inference

Michael Lui, Yavuz Yetim, Özgür Özkan +4

Deep learning recommendation models have grown to the terabyte scale. Traditional serving schemes--that load entire models to a single server--are unable to support this scale. One…