Learning Type-Aware Embeddings for Fashion Compatibility

Abstract

Outfits in online fashion data are composed of items of many different types (e.g. top, bottom, shoes) that share some stylistic relationship with one another. A representation for building outfits requires a method that can learn both notions of similarity (for example, when two tops are interchangeable) and compatibility (items of possibly different type that can go together in an outfit). This paper presents an approach to learning an image embedding that respects item type, and jointly learns notions of item similarity and compatibility in an end-to-end model. To evaluate the learned representation, we crawled 68,306 outfits created by users on the Polyvore website. Our approach obtains 3-5% improvement over the state-of-the-art on outfit compatibility prediction and fill-in-the-blank tasks using our dataset, as well as an established smaller dataset, while supporting a variety of useful queries.

Accepted at ECCV 2018

References in corpus (7)

• Efficient Estimation of Word Representations in Vector Space
• Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Images
• Fisher Vectors Derived from Hybrid Gaussian-Laplacian Mixture Models for Image Annotation
• Be Your Own Prada: Fashion Synthesis with Structural Coherence
• Learning Visual Clothing Style with Heterogeneous Dyadic Co-occurrences
• Learning Compatibility Across Categories for Heterogeneous Item Recommendation
• Fashion Forward: Forecasting Visual Style in Fashion

Cited by in corpus (11)

• Fashion Meets Computer Vision: A Survey
• Exchangeable deep neural networks for set-to-set matching and learning
• Bootstrapping Complete The Look at Pinterest
• Context-Aware Visual Compatibility Prediction
• fAshIon after fashion: A Report of AI in Fashion
• Fashion Recommendation and Compatibility Prediction Using Relational Network
• Complete the Look: Scene-based Complementary Product Recommendation
• Learning Tuple Compatibility for Conditional OutfitRecommendation
• TransNFCM: Translation-Based Neural Fashion Compatibility Modeling
• Fine-Grained Fashion Similarity Learning by Attribute-Specific Embedding Network
• POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion