Graph Convolutional Neural Networks for Web-Scale Recommender Systems

Abstract

Recent advancements in deep neural networks for graph-structured data have led to state-of-the-art performance on recommender system benchmarks. However, making these methods practical and scalable to web-scale recommendation tasks with billions of items and hundreds of millions of users remains a challenge. Here we describe a large-scale deep recommendation engine that we developed and deployed at Pinterest. We develop a data-efficient Graph Convolutional Network (GCN) algorithm PinSage, which combines efficient random walks and graph convolutions to generate embeddings of nodes (i.e., items) that incorporate both graph structure as well as node feature information. Compared to prior GCN approaches, we develop a novel method based on highly efficient random walks to structure the convolutions and design a novel training strategy that relies on harder-and-harder training examples to improve robustness and convergence of the model. We also develop an efficient MapReduce model inference algorithm to generate embeddings using a trained model. We deploy PinSage at Pinterest and train it on 7.5 billion examples on a graph with 3 billion nodes representing pins and boards, and 18 billion edges. According to offline metrics, user studies and A/B tests, PinSage generates higher-quality recommendations than comparable deep learning and graph-based alternatives. To our knowledge, this is the largest application of deep graph embeddings to date and paves the way for a new generation of web-scale recommender systems based on graph convolutional architectures.

arXiv commentKDD 2018

References in corpus (4)

• DeepWalk: Online Learning of Social Representations
• Geometric deep learning: going beyond Euclidean data
• Molecular Graph Convolutions: Moving Beyond Fingerprints
• Ask the GRU: Multi-Task Learning for Deep Text Recommendations

Cited by in corpus (259)

• A Comprehensive Survey on Graph Neural Networks
• Neural Graph Collaborative Filtering
• Deep Learning based Recommender System: A Survey and New Perspectives
• Self-supervised Graph Learning for Recommendation
• Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
• Knowledge Graph Convolutional Networks for Recommender Systems
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training
• Disentangled Graph Collaborative Filtering
• Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning
• AM-GCN: Adaptive Multi-channel Graph Convolutional Networks
• Anomaly Detection on Attributed Networks via Contrastive Self-Supervised Learning
• Hypergraph Contrastive Collaborative Filtering
• Recommender Systems in the Era of Large Language Models (LLMs)
• A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation
• Scaling Graph Neural Networks with Approximate PageRank
• Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey
• FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation
• Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
• How to Build a Graph-Based Deep Learning Architecture in Traffic Domain: A Survey
• Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System
• Dual Graph Attention Networks for Deep Latent Representation of Multifaceted Social Effects in Recommender Systems
• Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
• Graph Representation Learning: A Survey
• Adaptive Graph Encoder for Attributed Graph Embedding
• Spam Review Detection with Graph Convolutional Networks
• Multi-level Cross-view Contrastive Learning for Knowledge-aware Recommender System
• Poisoning Attacks to Graph-Based Recommender Systems
• DeepGCNs: Making GCNs Go as Deep as CNNs
• ETA Prediction with Graph Neural Networks in Google Maps
• Combinatorial Optimization with Physics-Inspired Graph Neural Networks
• Contrastive Meta Learning with Behavior Multiplicity for Recommendation
• Stability Properties of Graph Neural Networks
• Federated Social Recommendation with Graph Neural Network
• Self-Supervised Hypergraph Transformer for Recommender Systems
• IntentGC: a Scalable Graph Convolution Framework Fusing Heterogeneous Information for Recommendation
• Graph Enhanced Representation Learning for News Recommendation
• APAN: Asynchronous Propagation Attention Network for Real-time Temporal Graph Embedding
• Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer
• DGRec: Graph Neural Network for Recommendation with Diversified Embedding Generation
• MCNE: An End-to-End Framework for Learning Multiple Conditional Network Representations of Social Network
• DGCN: Diversified Recommendation with Graph Convolutional Networks
• Differentiable Graph Module (DGM) for Graph Convolutional Networks
• Estimating Node Importance in Knowledge Graphs Using Graph Neural Networks
• Learning on Attribute-Missing Graphs
• Automated Self-Supervised Learning for Recommendation
• GraphACT: Accelerating GCN Training on CPU-FPGA Heterogeneous Platforms
• Graph Convolutional Networks for Graphs Containing Missing Features
• Semantic Models for the First-stage Retrieval: A Comprehensive Review
• Social Recommendation with Self-Supervised Metagraph Informax Network
• ASGN: An Active Semi-supervised Graph Neural Network for Molecular Property Prediction
• PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterest
• NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization
• GA2MIF: Graph and Attention Based Two-Stage Multi-Source Information Fusion for Conversational Emotion Detection
• Trustworthy Graph Neural Networks: Aspects, Methods and Trends
• Learning to Recommend with Multiple Cascading Behaviors
• CrossCBR: Cross-view Contrastive Learning for Bundle Recommendation
• GAG: Global Attributed Graph Neural Network for Streaming Session-based Recommendation
• KRED: Knowledge-Aware Document Representation for News Recommendations
• SelfCF: A Simple Framework for Self-supervised Collaborative Filtering
• Investigating and Mitigating Degree-Related Biases in Graph Convolutional Networks
• Improving Knowledge-aware Recommendation with Multi-level Interactive Contrastive Learning
• A Review of Knowledge Graph Completion
• Heterogeneous Graph Representation Learning with Relation Awareness
• A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability
• Solving Cold Start Problem in Recommendation with Attribute Graph Neural Networks
• TDGIA:Effective Injection Attacks on Graph Neural Networks
• Graph Transformer for Recommendation
• M2GRL: A Multi-task Multi-view Graph Representation Learning Framework for Web-scale Recommender Systems
• Graph Convolution Machine for Context-aware Recommender System
• Multi-Behavior Enhanced Recommendation with Cross-Interaction Collaborative Relation Modeling
• TransGCN:Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction
• Breaking the Limit of Graph Neural Networks by Improving the Assortativity of Graphs with Local Mixing Patterns
• Multi-Behavior Sequential Recommendation with Temporal Graph Transformer
• GraphSAIL: Graph Structure Aware Incremental Learning for Recommender Systems
• HRCF: Enhancing Collaborative Filtering via Hyperbolic Geometric Regularization
• Unnoticeable Backdoor Attacks on Graph Neural Networks
• On the Effectiveness of Sampled Softmax Loss for Item Recommendation
• Multi-Behavior Graph Neural Networks for Recommender System
• Collaboration-Aware Graph Convolutional Network for Recommender Systems
• Explainability in Music Recommender Systems
• Investigating Accuracy-Novelty Performance for Graph-based Collaborative Filtering
• The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey
• MGDCF: Distance Learning via Markov Graph Diffusion for Neural Collaborative Filtering
• On Representation Knowledge Distillation for Graph Neural Networks
• Machine Learning for Practical Quantum Error Mitigation
• Large-scale Personalized Video Game Recommendation via Social-aware Contextualized Graph Neural Network
• Accelerating Large Scale Real-Time GNN Inference using Channel Pruning
• Graph Neural Networks for Pressure Estimation in Water Distribution Systems
• Time Interval-enhanced Graph Neural Network for Shared-account Cross-domain Sequential Recommendation
• Invariance-Preserving Localized Activation Functions for Graph Neural Networks
• Graph-less Collaborative Filtering
• HICF: Hyperbolic Informative Collaborative Filtering
• TaxoExpan: Self-supervised Taxonomy Expansion with Position-Enhanced Graph Neural Network
• GIF: A General Graph Unlearning Strategy via Influence Function
• Joint Multi-grained Popularity-aware Graph Convolution Collaborative Filtering for Recommendation
• A Comparative Study for Unsupervised Network Representation Learning
• DynaMo: Dynamic Community Detection by Incrementally Maximizing Modularity
• HS-GCN: Hamming Spatial Graph Convolutional Networks for Recommendation
• MEGCF: Multimodal Entity Graph Collaborative Filtering for Personalized Recommendation
• Bring Your Own View: Graph Neural Networks for Link Prediction with Personalized Subgraph Selection
• C-SAW: A Framework for Graph Sampling and Random Walk on GPUs
• Beyond Low-pass Filtering: Graph Convolutional Networks with Automatic Filtering
• Click-Through Rate Prediction with Multi-Modal Hypergraphs
• Comprehensive Fair Meta-learned Recommender System
• Towards Semi-supervised Universal Graph Classification
• Scalable Graph Neural Network Training: The Case for Sampling
• PaSca: a Graph Neural Architecture Search System under the Scalable Paradigm
• TwHIN: Embedding the Twitter Heterogeneous Information Network for Personalized Recommendation
• A Survey on Graph Counterfactual Explanations: Definitions, Methods, Evaluation, and Research Challenges
• Indirect Adversarial Attacks via Poisoning Neighbors for Graph Convolutional Networks
• Ginex: SSD-enabled Billion-scale Graph Neural Network Training on a Single Machine via Provably Optimal In-memory Caching
• Predicting Biomedical Interactions with Higher-Order Graph Convolutional Networks
• HP-GNN: Generating High Throughput GNN Training Implementation on CPU-FPGA Heterogeneous Platform
• Formalizing Multimedia Recommendation through Multimodal Deep Learning
• Multi-Graph Convolutional Network for Relationship-Driven Stock Movement Prediction
• CenGCN: Centralized Convolutional Networks with Vertex Imbalance for Scale-Free Graphs
• STGlow: A Flow-based Generative Framework with Dual Graphormer for Pedestrian Trajectory Prediction
• FIGRET: Fine-Grained Robustness-Enhanced Traffic Engineering
• Carousel Personalization in Music Streaming Apps with Contextual Bandits
• Dynamically Expandable Graph Convolution for Streaming Recommendation
• SIGformer: Sign-aware Graph Transformer for Recommendation
• Medical Entity Disambiguation Using Graph Neural Networks
• CatGCN: Graph Convolutional Networks with Categorical Node Features
• Peer-inspired Student Performance Prediction in Interactive Online Question Pools with Graph Neural Network
• Knowing your FATE: Friendship, Action and Temporal Explanations for User Engagement Prediction on Social Apps
• Approximating Network Centrality Measures Using Node Embedding and Machine Learning
• Is a Single Vector Enough? Exploring Node Polysemy for Network Embedding
• Path-Aware Graph Attention for HD Maps in Motion Prediction
• TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest
• paper2repo: GitHub Repository Recommendation for Academic Papers
• TaxoEnrich: Self-Supervised Taxonomy Completion via Structure-Semantic Representations
• Efficient Heterogeneous Graph Learning via Random Projection
• Adap-$\tau$: Adaptively Modulating Embedding Magnitude for Recommendation
• Graph Neural Network for Stress Predictions in Stiffened Panels Under Uniform Loading
• Algorithm and System Co-design for Efficient Subgraph-based Graph Representation Learning
• Stochastic Graph Neural Networks
• Causal Collaborative Filtering
• LGNN: A Context-aware Line Segment Detector
• A Privacy Preserving System for Movie Recommendations Using Federated Learning
• Relational Graph Learning for Grounded Video Description Generation
• DuETA: Traffic Congestion Propagation Pattern Modeling via Efficient Graph Learning for ETA Prediction at Baidu Maps
• From Data to Decisions: The Transformational Power of Machine Learning in Business Recommendations
• MugRep: A Multi-Task Hierarchical Graph Representation Learning Framework for Real Estate Appraisal
• Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation
• Bayesian graph convolutional neural networks via tempered MCMC
• Knowledge Enhanced Multi-intent Transformer Network for Recommendation
• Genetic Meta-Structure Search for Recommendation on Heterogeneous Information Network
• MultiImport: Inferring Node Importance in a Knowledge Graph from Multiple Input Signals
• FauxWard: A Graph Neural Network Approach to Fauxtography Detection Using Social Media Comments
• Online Disease Self-diagnosis with Inductive Heterogeneous Graph Convolutional Networks
• A Comprehensive Survey on Self-Supervised Learning for Recommendation
• LL-GNN: Low Latency Graph Neural Networks on FPGAs for High Energy Physics
• INMO: A Model-Agnostic and Scalable Module for Inductive Collaborative Filtering
• Temporal Aggregation and Propagation Graph Neural Networks for Dynamic Representation
• LightGNN: Simple Graph Neural Network for Recommendation
• Action Sequence Augmentation for Early Graph-based Anomaly Detection
• SCARA: Scalable Graph Neural Networks with Feature-Oriented Optimization
• A Network Science perspective of Graph Convolutional Networks: A survey
• Adaptive Collaborative Filtering with Personalized Time Decay Functions for Financial Product Recommendation
• Combining social relations and interaction data in Recommender System with Graph Convolution Collaborative Filtering
• LPFormer: An Adaptive Graph Transformer for Link Prediction
• Node Injection for Class-specific Network Poisoning
• A Unifying View of Explicit and Implicit Feature Maps of Graph Kernels
• Challenging the Myth of Graph Collaborative Filtering: a Reasoned and Reproducibility-driven Analysis
• Generating Usage-related Questions for Preference Elicitation in Conversational Recommender Systems
• Structure fusion based on graph convolutional networks for semi-supervised classification
• Turning Privacy-preserving Mechanisms against Federated Learning
• Collaborate to Adapt: Source-Free Graph Domain Adaptation via Bi-directional Adaptation
• SWAG: Item Recommendations using Convolutions on Weighted Graphs
• Simple and Efficient Partial Graph Adversarial Attack: A New Perspective
• Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs
• Counterfactual Learning on Graphs: A Survey
• Interpreting the Mechanism of Synergism for Drug Combinations Using Attention-Based Hierarchical Graph Pooling
• FairEGM: Fair Link Prediction and Recommendation via Emulated Graph Modification
• Code Recommendation for Open Source Software Developers
• Dimension Independent Mixup for Hard Negative Sample in Collaborative Filtering
• Recommending on graphs: a comprehensive review from a data perspective
• SketchNE: Embedding Billion-Scale Networks Accurately in One Hour
• Rethinking Personalized Ranking at Pinterest: An End-to-End Approach
• MSPipe: Efficient Temporal GNN Training via Staleness-Aware Pipeline
• Revisiting Neighborhood-based Link Prediction for Collaborative Filtering
• Accurate, Efficient and Scalable Training of Graph Neural Networks
• Scaling User Modeling: Large-scale Online User Representations for Ads Personalization in Meta
• Unify Local and Global Information for Top-$N$ Recommendation
• Scalable Verification of GNN-based Job Schedulers
• Cross-Network Learning with Partially Aligned Graph Convolutional Networks
• Learning and Optimization of Implicit Negative Feedback for Industrial Short-video Recommender System
• Robust Node Classification on Graphs: Jointly from Bayesian Label Transition and Topology-based Label Propagation
• Superpixels and Graph Convolutional Neural Networks for Efficient Detection of Nutrient Deficiency Stress from Aerial Imagery
• Graph Neural Bandits
• NudgeRank: Digital Algorithmic Nudging for Personalized Health
• GraphConfRec: A Graph Neural Network-Based Conference Recommender System
• Initialization Matters: Regularizing Manifold-informed Initialization for Neural Recommendation Systems
• Improving the Shortest Plank: Vulnerability-Aware Adversarial Training for Robust Recommender System
• MixRec: Individual and Collective Mixing Empowers Data Augmentation for Recommender Systems
• Modeling Dual Period-Varying Preferences for Takeaway Recommendation
• Adversarial learning for product recommendation
• Cooperative Retriever and Ranker in Deep Recommenders
• Network Embedding via Deep Prediction Model
• LightSAGE: Graph Neural Networks for Large Scale Item Retrieval in Shopee's Advertisement Recommendation
• Unveiling Global Interactive Patterns across Graphs: Towards Interpretable Graph Neural Networks
• CARL-G: Clustering-Accelerated Representation Learning on Graphs
• Debiasing Neighbor Aggregation for Graph Neural Network in Recommender Systems
• The Impact of Global Structural Information in Graph Neural Networks Applications
• Constant Time Graph Neural Networks
• FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale
• Efficient and Joint Hyperparameter and Architecture Search for Collaborative Filtering
• OmniSearchSage: Multi-Task Multi-Entity Embeddings for Pinterest Search
• Geometric graphs from data to aid classification tasks with graph convolutional networks
• Heterogeneous Graph Representation of Stiffened Panels with Non-Uniform Boundary Conditions and Loads
• Heterogeneity-aware Cross-school Electives Recommendation: a Hybrid Federated Approach
• CROLoss: Towards a Customizable Loss for Retrieval Models in Recommender Systems
• Adversarial Collaborative Filtering for Free
• Generating Topological Structure of Floorplans from Room Attributes
• Learning Stochastic Graph Neural Networks with Constrained Variance
• Representation Online Matters: Practical End-to-End Diversification in Search and Recommender Systems
• Message-Passing Monte Carlo: Generating low-discrepancy point sets via Graph Neural Networks
• Adaptive Backdoor Attacks with Reasonable Constraints on Graph Neural Networks
• Improvement Graph Convolution Collaborative Filtering with Weighted addition input
• A Comprehensive Benchmark on Spectral GNNs: The Impact on Efficiency, Memory, and Effectiveness
• GraTO: Graph Neural Network Framework Tackling Over-smoothing with Neural Architecture Search
• Rankformer: A Graph Transformer for Recommendation based on Ranking Objective
• Globally Interpretable Graph Learning via Distribution Matching
• Optimizing Diffusion Rate and Label Reliability in a Graph-Based Semi-supervised Classifier
• OmniSage: Large Scale, Multi-Entity Heterogeneous Graph Representation Learning
• Learning Graph Neural Networks using Exact Compression
• Accelerating Backward Aggregation in GCN Training with Execution Path Preparing on GPUs
• SpotV2Net: Multivariate Intraday Spot Volatility Forecasting via Vol-of-Vol-Informed Graph Attention Networks
• GraphScale: A Framework to Enable Machine Learning over Billion-node Graphs
• A Scalable and Efficient Signal Integration System for Job Matching
• Rolling Forward: Enhancing LightGCN with Causal Graph Convolution for Credit Bond Recommendation
• SearchGCN: Powering Embedding Retrieval by Graph Convolution Networks for E-Commerce Search
• Semantic Graph Neural Network with Multi-measure Learning for Semi-supervised Classification
• SF-SGL: Solver-Free Spectral Graph Learning from Linear Measurements
• XInsight: Revealing Model Insights for GNNs with Flow-based Explanations
• Multi-view Correlation-aware Network Traffic Detection on Flow Hypergraph
• LPS-GNN : Deploying Graph Neural Networks on Graphs with 100-Billion Edges
• Graph-Based Model-Agnostic Data Subsampling for Recommendation Systems
• Scaling Large-scale GNN Training to Thousands of Processors on CPU-based Supercomputers
• MixDec Sampling: A Soft Link-based Sampling Method of Graph Neural Network for Recommendation
• Learning Heuristics for Transit Network Design and Improvement with Deep Reinforcement Learning
• Synergizing Implicit and Explicit User Interests: A Multi-Embedding Retrieval Framework at Pinterest
• Towards Lightweight and Automated Representation Learning System for Networks
• Robust Training of Temporal GNNs using Nearest Neighbours based Hard Negatives
• printf: Preference Modeling Based on User Reviews with Item Images and Textual Information via Graph Learning
• Improving One-class Recommendation with Multi-tasking on Various Preference Intensities
• Hessian-aware Quantized Node Embeddings for Recommendation
• EfficientRec an unlimited user-item scale recommendation system based on clustering and users interaction embedding profile
• The Limits of Graph Samplers for Training Inductive Recommender Systems: Extended results
• Understanding the Design Principles of Link Prediction in Directed Settings
• Category-based and Popularity-guided Video Game Recommendation: A Balance-oriented Framework
• Generative Data Augmentation in Graph Contrastive Learning for Recommendation
• InteractRank: Personalized Web-Scale Search Pre-Ranking with Cross Interaction Features
• Dual prototype attentive graph network for cross-market recommendation
• A Survey on Graph Neural Networks for Fraud Detection in Ride Hailing Platforms
• Hierarchical graph sampling based minibatch learning with chain preservation and variance reduction
• Efficient Graph based Recommender System with Weighted Averaging of Messages
• Scalable Deep Metric Learning on Attributed Graphs
• Augmenting Knowledge Graph Hierarchies Using Neural Transformers