Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks

Abstract

Modeling sequential interactions between users and items/products is crucial in domains such as e-commerce, social networking, and education. Representation learning presents an attractive opportunity to model the dynamic evolution of users and items, where each user/item can be embedded in a Euclidean space and its evolution can be modeled by an embedding trajectory in this space. However, existing dynamic embedding methods generate embeddings only when users take actions and do not explicitly model the future trajectory of the user/item in the embedding space. Here we propose JODIE, a coupled recurrent neural network model that learns the embedding trajectories of users and items. JODIE employs two recurrent neural networks to update the embedding of a user and an item at every interaction. Crucially, JODIE also models the future embedding trajectory of a user/item. To this end, it introduces a novel projection operator that learns to estimate the embedding of the user at any time in the future. These estimated embeddings are then used to predict future user-item interactions. To make the method scalable, we develop a t-Batch algorithm that creates time-consistent batches and leads to 9x faster training. We conduct six experiments to validate JODIE on two prediction tasks---future interaction prediction and state change prediction---using four real-world datasets. We show that JODIE outperforms six state-of-the-art algorithms in these tasks by at least 20% in predicting future interactions and 12% in state change prediction.

arXiv commentACM SIGKDD 2019 research track oral paper. The code and datasets are available on the project website: https://github.com/srijankr/jodie/ arXiv admin note: substantial text overlap with arXiv:1812.02289

References in corpus (13)

• Graph Embedding Techniques, Applications, and Performance: A Survey
• Deep Learning based Recommender System: A Survey and New Perspectives
• Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec
• Anyone Can Become a Troll: Causes of Trolling Behavior in Online Discussions
• Attributed Network Embedding for Learning in a Dynamic Environment
• Community Interaction and Conflict on the Web
• Representation Learning on Graphs: Methods and Applications
• Scalable Link Prediction in Dynamic Networks via Non-Negative Matrix Factorization
• Fast ALS-based tensor factorization for context-aware recommendation from implicit feedback
• Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs
• VEWS: A Wikipedia Vandal Early Warning System
• COEVOLVE: A Joint Point Process Model for Information Diffusion and Network Co-evolution
• DyLink2Vec: Effective Feature Representation for Link Prediction in Dynamic Networks

Cited by in corpus (56)

• Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey
• APAN: Asynchronous Propagation Attention Network for Real-time Temporal Graph Embedding
• Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer
• Graph Neural Networks for Financial Fraud Detection: A Review
• Lightweight Self-Attentive Sequential Recommendation
• Self-Supervised Temporal Graph learning with Temporal and Structural Intensity Alignment
• EvoKG: Jointly Modeling Event Time and Network Structure for Reasoning over Temporal Knowledge Graphs
• TREND: TempoRal Event and Node Dynamics for Graph Representation Learning
• The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey
• FairSR: Fairness-aware Sequential Recommendation through Multi-Task Learning with Preference Graph Embeddings
• Continual Learning on Dynamic Graphs via Parameter Isolation
• Continuous-Time and Multi-Level Graph Representation Learning for Origin-Destination Demand Prediction
• Evolutionary Preference Learning via Graph Nested GRU ODE for Session-based Recommendation
• Dynamically Expandable Graph Convolution for Streaming Recommendation
• TMac: Temporal Multi-Modal Graph Learning for Acoustic Event Classification
• Deep learning for dynamic graphs: models and benchmarks
• DyTed: Disentangled Representation Learning for Discrete-time Dynamic Graph
• Empowering Next POI Recommendation with Multi-Relational Modeling
• Graph Sequential Neural ODE Process for Link Prediction on Dynamic and Sparse Graphs
• Conversations Gone Alright: Quantifying and Predicting Prosocial Outcomes in Online Conversations
• Subset Node Anomaly Tracking over Large Dynamic Graphs
• Deep Temporal Graph Clustering: A Comprehensive Benchmark and Datasets
• A Self-supervised Riemannian GNN with Time Varying Curvature for Temporal Graph Learning
• Rank List Sensitivity of Recommender Systems to Interaction Perturbations
• CLDG: Contrastive Learning on Dynamic Graphs
• Temporal Aggregation and Propagation Graph Neural Networks for Dynamic Representation
• Transition Propagation Graph Neural Networks for Temporal Networks
• Action Sequence Augmentation for Early Graph-based Anomaly Detection
• Predicting Information Pathways Across Online Communities
• Understanding the network formation pattern for better link prediction
• Dynamic Graph Unlearning: A General and Efficient Post-Processing Method via Gradient Transformation
• MSPipe: Efficient Temporal GNN Training via Staleness-Aware Pipeline
• Link prediction in dynamic networks using random dot product graphs
• TGFormer: Towards Temporal Graph Transformer with Auto-Correlation Mechanism
• Temporal Graph Representation Learning with Adaptive Augmentation Contrastive
• EasyDGL: Encode, Train and Interpret for Continuous-time Dynamic Graph Learning
• Implicit Session Contexts for Next-Item Recommendations
• PopNet: Real-Time Population-Level Disease Prediction with Data Latency
• CPMR: Context-Aware Incremental Sequential Recommendation with Pseudo-Multi-Task Learning
• Graph Evidential Learning for Anomaly Detection
• Missing Data Estimation in Temporal Multilayer Position-aware Graph Neural Network (TMP-GNN)
• Modeling Inter-Dependence Between Time and Mark in Multivariate Temporal Point Processes
• Non-exchangeable Conformal Prediction for Temporal Graph Neural Networks
• Dynamic Graph Representation Learning for Passenger Behavior Prediction
• Dynamic Mixed Membership Stochastic Block Model for Weighted Labeled Networks
• TIDFormer: Exploiting Temporal and Interactive Dynamics Makes A Great Dynamic Graph Transformer
• ForeSeer: Product Aspect Forecasting Using Temporal Graph Embedding
• A Frequency-Structure Approach for Link Stream Analysis
• Robust Training of Temporal GNNs using Nearest Neighbours based Hard Negatives
• SEP-GCN: Leveraging Similar Edge Pairs with Temporal and Spatial Contexts for Location-Based Recommender Systems
• Learning Feature Encoder with Synthetic Anomalies for Weakly Supervised Graph Anomaly Detection
• SimHawNet: A Modified Hawkes Process for Temporal Network Simulation
• Leveraging Vulnerabilities in Temporal Graph Neural Networks via Strategic High-Impact Assaults
• Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation
• Evaluating link prediction: New perspectives and recommendations
• A Transfer Framework for Enhancing Temporal Graph Learning in Data-Scarce Settings