Improving Outfit Recommendation with Co-supervision of Fashion Generation

Abstract

The task of fashion recommendation includes two main challenges: visual understanding and visual matching. Visual understanding aims to extract effective visual features. Visual matching aims to model a human notion of compatibility to compute a match between fashion items. Most previous studies rely on recommendation loss alone to guide visual understanding and matching. Although the features captured by these methods describe basic characteristics (e.g., color, texture, shape) of the input items, they are not directly related to the visual signals of the output items (to be recommended). This is problematic because the aesthetic characteristics (e.g., style, design), based on which we can directly infer the output items, are lacking. Features are learned under the recommendation loss alone, where the supervision signal is simply whether the given two items are matched or not. To address this problem, we propose a neural co-supervision learning framework, called the FAshion Recommendation Machine (FARM). FARM improves visual understanding by incorporating the supervision of generation loss, which we hypothesize to be able to better encode aesthetic information. FARM enhances visual matching by introducing a novel layer-to-layer matching mechanism to fuse aesthetic information more effectively, and meanwhile avoiding paying too much attention to the generation quality and ignoring the recommendation performance. Extensive experiments on two publicly available datasets show that FARM outperforms state-of-the-art models on outfit recommendation, in terms of AUC and MRR. Detailed analyses of generated and recommended items demonstrate that FARM can encode better features and generate high quality images as references to improve recommendation performance.

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Auto-Encoding Variational Bayes
• BPR: Bayesian Personalized Ranking from Implicit Feedback
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Variational Inference: A Review for Statisticians
• On the difficulty of training Recurrent Neural Networks
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Learning Fashion Compatibility with Bidirectional LSTMs
• Mining Fashion Outfit Composition Using An End-to-End Deep Learning Approach on Set Data
• Aesthetic-based Clothing Recommendation
• Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation
• CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training
• Outfit Generation and Style Extraction via Bidirectional LSTM and Autoencoder
• Modeling Visual Compatibility through Hierarchical Mid-level Elements

Cited by in corpus (5)

• Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation
• Exploring AI in Fashion: A Review of Aesthetics, Personalization, Virtual Try-On, and Forecasting
• Exploiting Latent Codes: Interactive Fashion Product Generation, Similar Image Retrieval, and Cross-Category Recommendation using Variational Autoencoders
• Learning Tuple Compatibility for Conditional OutfitRecommendation
• Searching Personal Collections