Self-Attentive Sequential Recommendation

Abstract

Sequential dynamics are a key feature of many modern recommender systems, which seek to capture the `context' of users' activities on the basis of actions they have performed recently. To capture such patterns, two approaches have proliferated: Markov Chains (MCs) and Recurrent Neural Networks (RNNs). Markov Chains assume that a user's next action can be predicted on the basis of just their last (or last few) actions, while RNNs in principle allow for longer-term semantics to be uncovered. Generally speaking, MC-based methods perform best in extremely sparse datasets, where model parsimony is critical, while RNNs perform better in denser datasets where higher model complexity is affordable. The goal of our work is to balance these two goals, by proposing a self-attention based sequential model (SASRec) that allows us to capture long-term semantics (like an RNN), but, using an attention mechanism, makes its predictions based on relatively few actions (like an MC). At each time step, SASRec seeks to identify which items are `relevant' from a user's action history, and use them to predict the next item. Extensive empirical studies show that our method outperforms various state-of-the-art sequential models (including MC/CNN/RNN-based approaches) on both sparse and dense datasets. Moreover, the model is an order of magnitude more efficient than comparable CNN/RNN-based models. Visualizations on attention weights also show how our model adaptively handles datasets with various density, and uncovers meaningful patterns in activity sequences.

arXiv commentAccepted by ICDM'18 as a long paper

References in corpus (10)

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Recurrent Neural Networks with Top-k Gains for Session-based Recommendations
• Translation-based Recommendation
• Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks
• Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation
• An empirical analysis of dropout in piecewise linear networks
• Visually-Aware Fashion Recommendation and Design with Generative Image Models
• Context-aware Sequential Recommendation

Cited by in corpus (20)

• MEANTIME: Mixture of Attention Mechanisms with Multi-temporal Embeddings for Sequential Recommendation
• Deep Session Interest Network for Click-Through Rate Prediction
• Stochastic Shared Embeddings: Data-driven Regularization of Embedding Layers
• Self-Supervised Reinforcement Learning for Recommender Systems
• SANST: A Self-Attentive Network for Next Point-of-Interest Recommendation
• Personalized Re-ranking for Recommendation
• Modeling Personalized Item Frequency Information for Next-basket Recommendation
• Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation
• Initialization Matters: Regularizing Manifold-informed Initialization for Neural Recommendation Systems
• Inter-sequence Enhanced Framework for Personalized Sequential Recommendation
• Denoising User-aware Memory Network for Recommendation
• Memory Augmented Neural Model for Incremental Session-based Recommendation
• Interpretable ICD Code Embeddings with Self- and Mutual-Attention Mechanisms
• Sequential Movie Genre Prediction using Average Transition Probability with Clustering
• Temporal Collaborative Ranking Via Personalized Transformer
• Social Influence-based Attentive Mavens Mining and Aggregative Representation Learning for Group Recommendation
• Memory Augmented Graph Neural Networks for Sequential Recommendation
• Advances in Collaborative Filtering and Ranking
• Breaking the Softmax Bottleneck for Sequential Recommender Systems with Dropout and Decoupling
• Freudian and Newtonian Recurrent Cell for Sequential Recommendation