Visually-Aware Fashion Recommendation and Design with Generative Image Models

Abstract

Building effective recommender systems for domains like fashion is challenging due to the high level of subjectivity and the semantic complexity of the features involved (i.e., fashion styles). Recent work has shown that approaches to `visual' recommendation (e.g.~clothing, art, etc.) can be made more accurate by incorporating visual signals directly into the recommendation objective, using `off-the-shelf' feature representations derived from deep networks. Here, we seek to extend this contribution by showing that recommendation performance can be significantly improved by learning `fashion aware' image representations directly, i.e., by training the image representation (from the pixel level) and the recommender system jointly; this contribution is related to recent work using Siamese CNNs, though we are able to show improvements over state-of-the-art recommendation techniques such as BPR and variants that make use of pre-trained visual features. Furthermore, we show that our model can be used \emph{generatively}, i.e., given a user and a product category, we can generate new images (i.e., clothing items) that are most consistent with their personal taste. This represents a first step towards building systems that go beyond recommending existing items from a product corpus, but which can be used to suggest styles and aid the design of new products.

arXiv comment10 pages, 6 figures. Accepted by ICDM'17 as a long paper

References in corpus (8)

• Conditional Generative Adversarial Nets
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
• Least Squares Generative Adversarial Networks
• Metadata Embeddings for User and Item Cold-start Recommendations
• Fashion Forward: Forecasting Visual Style in Fashion

Cited by in corpus (8)

• Self-Attentive Sequential Recommendation
• Towards Explainable Personalized Recommendations by Learning from Users' Photos
• CnGAN: Generative Adversarial Networks for Cross-network user preference generation for non-overlapped users
• Explainable Fashion Recommendation: A Semantic Attribute Region Guided Approach
• Personalizing Similar Product Recommendations in Fashion E-commerce
• Sentiment Analysis of Fashion Related Posts in Social Media
• Exploring AI in Fashion: A Review of Aesthetics, Personalization, Virtual Try-On, and Forecasting
• Recommendation Through Mixtures of Heterogeneous Item Relationships