Learning Fashion Compatibility with Bidirectional LSTMs

Abstract

The ubiquity of online fashion shopping demands effective recommendation services for customers. In this paper, we study two types of fashion recommendation: (i) suggesting an item that matches existing components in a set to form a stylish outfit (a collection of fashion items), and (ii) generating an outfit with multimodal (images/text) specifications from a user. To this end, we propose to jointly learn a visual-semantic embedding and the compatibility relationships among fashion items in an end-to-end fashion. More specifically, we consider a fashion outfit to be a sequence (usually from top to bottom and then accessories) and each item in the outfit as a time step. Given the fashion items in an outfit, we train a bidirectional LSTM (Bi-LSTM) model to sequentially predict the next item conditioned on previous ones to learn their compatibility relationships. Further, we learn a visual-semantic space by regressing image features to their semantic representations aiming to inject attribute and category information as a regularization for training the LSTM. The trained network can not only perform the aforementioned recommendations effectively but also predict the compatibility of a given outfit. We conduct extensive experiments on our newly collected Polyvore dataset, and the results provide strong qualitative and quantitative evidence that our framework outperforms alternative methods.

ACM MM 17

References in corpus (1)

• Clothing Co-Parsing by Joint Image Segmentation and Labeling

Cited by in corpus (22)

• Explainable Outfit Recommendation with Joint Outfit Matching and Comment Generation
• Dressing as a Whole: Outfit Compatibility Learning Based on Node-wise Graph Neural Networks
• Improving Outfit Recommendation with Co-supervision of Fashion Generation
• COutfitGAN: Learning to Synthesize Compatible Outfits Supervised by Silhouette Masks and Fashion Styles
• Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning
• Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network
• SGDiff: A Style Guided Diffusion Model for Fashion Synthesis
• Formalizing Multimedia Recommendation through Multimodal Deep Learning
• Learning to Synthesize Compatible Fashion Items Using Semantic Alignment and Collocation Classification: An Outfit Generation Framework
• CausalRec: Causal Inference for Visual Debiasing in Visually-Aware Recommendation
• BC-GAN: A Generative Adversarial Network for Synthesizing a Batch of Collocated Clothing
• Hierarchical Matching and Reasoning for Multi-Query Image Retrieval
• Addressing the Cold-Start Problem in Outfit Recommendation Using Visual Preference Modelling
• Exchangeable deep neural networks for set-to-set matching and learning
• Sentiment Analysis of Fashion Related Posts in Social Media
• Limb-Aware Virtual Try-On Network with Progressive Clothing Warping
• Exploring AI in Fashion: A Review of Aesthetics, Personalization, Virtual Try-On, and Forecasting
• MMFL-Net: Multi-scale and Multi-granularity Feature Learning for Cross-domain Fashion Retrieval
• Attribute-Guided Multi-Level Attention Network for Fine-Grained Fashion Retrieval
• Dual Preference Distribution Learning for Item Recommendation
• Main Product Detection with Graph Networks for Fashion
• An Application to Generate Style Guided Compatible Outfit