The Architectural Implications of Facebook's DNN-based Personalized Recommendation

Abstract

The widespread application of deep learning has changed the landscape of computation in the data center. In particular, personalized recommendation for content ranking is now largely accomplished leveraging deep neural networks. However, despite the importance of these models and the amount of compute cycles they consume, relatively little research attention has been devoted to systems for recommendation. To facilitate research and to advance the understanding of these workloads, this paper presents a set of real-world, production-scale DNNs for personalized recommendation coupled with relevant performance metrics for evaluation. In addition to releasing a set of open-source workloads, we conduct in-depth analysis that underpins future system design and optimization for at-scale recommendation: Inference latency varies by 60% across three Intel server generations, batching and co-location of inferences can drastically improve latency-bounded throughput, and the diverse composition of recommendation models leads to different optimization strategies.

arXiv comment11 pages

References in corpus (12)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Deep Learning Recommendation Model for Personalization and Recommendation Systems
• Fathom: Reference Workloads for Modern Deep Learning Methods
• MLPerf Training Benchmark
• Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
• Exploring Neural Transducers for End-to-End Speech Recognition
• Bandana: Using Non-volatile Memory for Storing Deep Learning Models
• On the Dimensionality of Embeddings for Sparse Features and Data

Cited by in corpus (7)

• SMASH: Co-designing Software Compression and Hardware-Accelerated Indexing for Efficient Sparse Matrix Operations
• DeepRecSys: A System for Optimizing End-To-End At-scale Neural Recommendation Inference
• Developing a Recommendation Benchmark for MLPerf Training and Inference
• RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
• AI Tax: The Hidden Cost of AI Data Center Applications
• RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference
• A Social Search Model for Large Scale Social Networks