Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time

Abstract

User experience in modern content discovery applications critically depends on high-quality personalized recommendations. However, building systems that provide such recommendations presents a major challenge due to a massive pool of items, a large number of users, and requirements for recommendations to be responsive to user actions and generated on demand in real-time. Here we present Pixie, a scalable graph-based real-time recommender system that we developed and deployed at Pinterest. Given a set of user-specific pins as a query, Pixie selects in real-time from billions of possible pins those that are most related to the query. To generate recommendations, we develop Pixie Random Walk algorithm that utilizes the Pinterest object graph of 3 billion nodes and 17 billion edges. Experiments show that recommendations provided by Pixie lead up to 50% higher user engagement when compared to the previous Hadoop-based production system. Furthermore, we develop a graph pruning strategy at that leads to an additional 58% improvement in recommendations. Last, we discuss system aspects of Pixie, where a single server executes 1,200 recommendation requests per second with 60 millisecond latency. Today, systems backed by Pixie contribute to more than 80% of all user engagement on Pinterest.

References in corpus (4)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• Wide & Deep Learning for Recommender Systems
• Joint Deep Modeling of Users and Items Using Reviews for Recommendation

Cited by in corpus (10)

• DGCN: Diversified Recommendation with Graph Convolutional Networks
• PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterest
• PRINCE: Provider-side Interpretability with Counterfactual Explanations in Recommender Systems
• An analysis of the graph processing landscape
• Revisiting Neural Retrieval on Accelerators
• Query-dominant User Interest Network for Large-Scale Search Ranking
• Frame invariance and scalability of neural operators for partial differential equations
• On Gradient Boosted Decision Trees and Neural Rankers: A Case-Study on Short-Video Recommendations at ShareChat
• Equivariant Entity-Relationship Networks
• Evolution of a Web-Scale Near Duplicate Image Detection System