RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
arXiv:1912.12953
Abstract
Personalized recommendation systems leverage deep learning models and account for the majority of data center AI cycles. Their performance is dominated by memory-bound sparse embedding operations with unique irregular memory access patterns that pose a fundamental challenge to accelerate. This paper proposes a lightweight, commodity DRAM compliant, near-memory processing solution to accelerate personalized recommendation inference. The in-depth characterization of production-grade recommendation models shows that embedding operations with high model-, operator- and data-level parallelism lead to memory bandwidth saturation, limiting recommendation inference performance. We propose RecNMP which provides a scalable solution to improve system throughput, supporting a broad range of sparse embedding models. RecNMP is specifically tailored to production environments with heavy co-location of operators on a single server. Several hardware/software co-optimization techniques such as memory-side caching, table-aware packet scheduling, and hot entry profiling are studied, resulting in up to 9.8x memory latency speedup over a highly-optimized baseline. Overall, RecNMP offers 4.2x throughput improvement and 45.8% memory energy savings.
References in corpus (7)
- Deep Learning Recommendation Model for Personalization and Recommendation Systems
- Basic Performance Measurements of the Intel Optane DC Persistent Memory Module
- Fathom: Reference Workloads for Modern Deep Learning Methods
- Bandana: Using Non-volatile Memory for Storing Deep Learning Models
- DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction
- The Architectural Implications of Facebook's DNN-based Personalized Recommendation
- Competitive Analysis System for Theatrical Movie Releases Based on Movie Trailer Deep Video Representation