Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation

Abstract

Most previous work on outfit recommendation focuses on designing visual features to enhance recommendations. Existing work neglects user comments of fashion items, which have been proved to be effective in generating explanations along with better recommendation results. We propose a novel neural network framework, neural outfit recommendation (NOR), that simultaneously provides outfit recommendations and generates abstractive comments. NOR consists of two parts: outfit matching and comment generation. For outfit matching, we propose a convolutional neural network with a mutual attention mechanism to extract visual features. The visual features are then decoded into a rating score for the matching prediction. For abstractive comment generation, we propose a gated recurrent neural network with a cross-modality attention mechanism to transform visual features into a concise sentence. The two parts are jointly trained based on a multi-task learning framework in an end-to-end back-propagation paradigm. Extensive experiments conducted on an existing dataset and a collected real-world dataset show NOR achieves significant improvements over state-of-the-art baselines for outfit recommendation. Meanwhile, our generated comments achieve impressive ROUGE and BLEU scores in comparison to human-written comments. The generated comments can be regarded as explanations for the recommendation results. We release the dataset and code to facilitate future research.

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Neural Machine Translation by Jointly Learning to Align and Translate
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• Caffe: Convolutional Architecture for Fast Feature Embedding
• On the difficulty of training Recurrent Neural Networks
• LexRank: Graph-based Lexical Centrality as Salience in Text Summarization
• Learning to Generate Reviews and Discovering Sentiment
• Learning Fashion Compatibility with Bidirectional LSTMs
• Neural Rating Regression with Abstractive Tips Generation for Recommendation
• Selective Encoding for Abstractive Sentence Summarization
• Explainable Recommendation via Multi-Task Learning in Opinionated Text Data
• Mining Fashion Outfit Composition Using An End-to-End Deep Learning Approach on Set Data
• Context-aware Natural Language Generation with Recurrent Neural Networks
• Improving Outfit Recommendation with Co-supervision of Fashion Generation

Cited by in corpus (12)

• Improving Outfit Recommendation with Co-supervision of Fashion Generation
• $c^{+}$GAN: Complementary Fashion Item Recommendation
• Impact of Tone-Aware Explanations in Recommender Systems
• BOXREC: Recommending a Box of Preferred Outfits in Online Shopping
• Exploring AI in Fashion: A Review of Aesthetics, Personalization, Virtual Try-On, and Forecasting
• Learning Color Compatibility in Fashion Outfits
• Regularizing Reasons for Outfit Evaluation with Gradient Penalty
• Fashion Recommendation and Compatibility Prediction Using Relational Network
• Learning Tuple Compatibility for Conditional OutfitRecommendation
• Garment Design with Generative Adversarial Networks
• Category Aware Explainable Conversational Recommendation
• Attribute-aware Explainable Complementary Clothing Recommendation