Evaluation of Session-based Recommendation Algorithms

Abstract

Recommender systems help users find relevant items of interest, for example on e-commerce or media streaming sites. Most academic research is concerned with approaches that personalize the recommendations according to long-term user profiles. In many real-world applications, however, such long-term profiles often do not exist and recommendations therefore have to be made solely based on the observed behavior of a user during an ongoing session. Given the high practical relevance of the problem, an increased interest in this problem can be observed in recent years, leading to a number of proposals for session-based recommendation algorithms that typically aim to predict the user's immediate next actions. In this work, we present the results of an in-depth performance comparison of a number of such algorithms, using a variety of datasets and evaluation measures. Our comparison includes the most recent approaches based on recurrent neural networks like GRU4REC, factorized Markov model approaches such as FISM or FOSSIL, as well as simpler methods based, e.g., on nearest neighbor schemes. Our experiments reveal that algorithms of this latter class, despite their sometimes almost trivial nature, often perform equally well or significantly better than today's more complex approaches based on deep neural networks. Our results therefore suggest that there is substantial room for improvement regarding the development of more sophisticated session-based recommendation algorithms.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• ADADELTA: An Adaptive Learning Rate Method
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• Recurrent Neural Networks with Top-k Gains for Session-based Recommendations
• Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks
• Learning from History and Present: Next-item Recommendation via Discriminatively Exploiting User Behaviors
• Meta-Prod2Vec - Product Embeddings Using Side-Information for Recommendation
• Sequential Click Prediction for Sponsored Search with Recurrent Neural Networks
• Personalized News Recommendation with Context Trees

Cited by in corpus (62)

• Deep Learning based Recommender System: A Survey and New Perspectives
• Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches
• Measuring the Business Value of Recommender Systems
• A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research
• Empirical Analysis of Session-Based Recommendation Algorithms
• Leveraging Large Language Models for Sequential Recommendation
• Personalized Graph Neural Networks with Attention Mechanism for Session-Aware Recommendation
• Contextual Hybrid Session-based News Recommendation with Recurrent Neural Networks
• News Session-Based Recommendations using Deep Neural Networks
• A Survey on Session-based Recommender Systems
• Session-aware Recommendation: A Surprising Quest for the State-of-the-art
• Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions
• Large Language Models for Intent-Driven Session Recommendations
• Diversification in Session-based News Recommender Systems
• Multi-intent-aware Session-based Recommendation
• Jointly Modeling Intra- and Inter-transaction Dependencies with Hierarchical Attentive Transaction Embeddings for Next-item Recommendation
• Predicting purchasing intent: Automatic Feature Learning using Recurrent Neural Networks
• Exploring Longitudinal Effects of Session-based Recommendations
• Widespread Flaws in Offline Evaluation of Recommender Systems
• Session-aware Linear Item-Item Models for Session-based Recommendation
• APGL4SR: A Generic Framework with Adaptive and Personalized Global Collaborative Information in Sequential Recommendation
• S-Walk: Accurate and Scalable Session-based Recommendationwith Random Walks
• Understanding Diversity in Session-Based Recommendation
• Predicting Information Pathways Across Online Communities
• Beyond NDCG: behavioral testing of recommender systems with RecList
• Critically Examining the Claimed Value of Convolutions over User-Item Embedding Maps for Recommender Systems
• A Reproducible Analysis of Sequential Recommender Systems
• Off-line vs. On-line Evaluation of Recommender Systems in Small E-commerce
• Sequential recommendation with metric models based on frequent sequences
• STAR: A Session-Based Time-Aware Recommender System
• Balancing Multi-level Interactions for Session-based Recommendation
• Are We Really Achieving Better Beyond-Accuracy Performance in Next Basket Recommendation?
• Time to Split: Exploring Data Splitting Strategies for Offline Evaluation of Sequential Recommenders
• Improving Sequential Recommendations with LLMs
• A Scalable Framework for Automatic Playlist Continuation on Music Streaming Services
• Multi-modal Embedding Fusion-based Recommender
• Modeling Personalized Item Frequency Information for Next-basket Recommendation
• Time is of the Essence: a Joint Hierarchical RNN and Point Process Model for Time and Item Predictions
• Context Tree for Adaptive Session-based Recommendation
• Learning Recommendations from User Actions in the Item-poor Insurance Domain
• Batch-Constrained Distributional Reinforcement Learning for Session-based Recommendation
• Performance Comparison of Session-based Recommendation Algorithms based on GNNs
• Semi-supervised Adversarial Learning for Complementary Item Recommendation
• Session-Based Hotel Recommendations: Challenges and Future Directions
• Sequential Recommendation in Online Games with Multiple Sequences, Tasks and User Levels
• New Item Consumption Prediction Using Deep Learning
• Using user's local context to support local news
• A Graph-based Method for Session-based Recommendations
• Neighborhood-Enhanced and Time-Aware Model for Session-based Recommendation
• Sequential modeling of Sessions using Recurrent Neural Networks for Skip Prediction
• Extending a Tag-based Collaborative Recommender with Co-occurring Information Interests
• ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation
• An Evaluation Study of Generative Adversarial Networks for Collaborative Filtering
• Exploring Global Information for Session-based Recommendation
• Linear Item-Item Model with Neural Knowledge for Session-based Recommendation
• Improving Next-Application Prediction with Deep Personalized-Attention Neural Network
• Recommending Target Actions Outside Sessions in the Data-poor Insurance Domain
• Many-to-one Recurrent Neural Network for Session-based Recommendation
• Hybrid Session-based News Recommendation using Recurrent Neural Networks
• Session-based k-NNs with Semantic Suggestions for Next-item Prediction
• Predicting Online Item-choice Behavior: A Shape-restricted Regression Perspective
• Next-item Recommendations in Short Sessions