Explainable Recommendation with Simulated Human Feedback

Abstract

Recent advancements in explainable recommendation have greatly bolstered user experience by elucidating the decision-making rationale. However, the existing methods actually fail to provide effective feedback signals for potentially better or worse generated explanations due to their reliance on traditional supervised learning paradigms in sparse interaction data. To address these issues, we propose a novel human-like feedback-driven optimization framework. This framework employs a dynamic interactive optimization mechanism for achieving human-centered explainable requirements without incurring high labor costs. Specifically, we propose to utilize large language models (LLMs) as human simulators to predict human-like feedback for guiding the learning process. To enable the LLMs to deeply understand the task essence and meet user's diverse personalized requirements, we introduce a human-induced customized reward scoring method, which helps stimulate the language understanding and logical reasoning capabilities of LLMs. Furthermore, considering the potential conflicts between different perspectives of explanation quality, we introduce a principled Pareto optimization that transforms the multi-perspective quality enhancement task into a multi-objective optimization problem for improving explanation performance. At last, to achieve efficient model training, we design an off-policy optimization pipeline. By incorporating a replay buffer and addressing the data distribution biases, we can effectively improve data utilization and enhance model generality. Extensive experiments on four datasets demonstrate the superiority of our approach.

References in corpus (12)

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• Learning over Knowledge-Base Embeddings for Recommendation
• Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation
• Reinforcement Knowledge Graph Reasoning for Explainable Recommendation
• Neural Rating Regression with Abstractive Tips Generation for Recommendation
• Explainable Recommendation via Multi-Task Learning in Opinionated Text Data
• Counterfactual Explainable Recommendation
• CAFE: Coarse-to-Fine Neural Symbolic Reasoning for Explainable Recommendation
• Neural Collaborative Reasoning
• RGRecSys: A Toolkit for Robustness Evaluation of Recommender Systems
• Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards
• On the Relationship between Explanation and Recommendation: Learning to Rank Explanations for Improved Performance