Learning Visual Clothing Style with Heterogeneous Dyadic Co-occurrences

Abstract

With the rapid proliferation of smart mobile devices, users now take millions of photos every day. These include large numbers of clothing and accessory images. We would like to answer questions like `What outfit goes well with this pair of shoes?' To answer these types of questions, one has to go beyond learning visual similarity and learn a visual notion of compatibility across categories. In this paper, we propose a novel learning framework to help answer these types of questions. The main idea of this framework is to learn a feature transformation from images of items into a latent space that expresses compatibility. For the feature transformation, we use a Siamese Convolutional Neural Network (CNN) architecture, where training examples are pairs of items that are either compatible or incompatible. We model compatibility based on co-occurrence in large-scale user behavior data; in particular co-purchase data from Amazon.com. To learn cross-category fit, we introduce a strategic method to sample training data, where pairs of items are heterogeneous dyads, i.e., the two elements of a pair belong to different high-level categories. While this approach is applicable to a wide variety of settings, we focus on the representative problem of learning compatible clothing style. Our results indicate that the proposed framework is capable of learning semantic information about visual style and is able to generate outfits of clothes, with items from different categories, that go well together.

ICCV 2015

References in corpus (3)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Going Deeper with Convolutions
• Material Recognition in the Wild with the Materials in Context Database

Cited by in corpus (26)

• Fashion-Gen: The Generative Fashion Dataset and Challenge
• Fashion Meets Computer Vision: A Survey
• Looking at Outfit to Parse Clothing
• Deep Interest Network for Click-Through Rate Prediction
• Style2Vec: Representation Learning for Fashion Items from Style Sets
• Learning Type-Aware Embeddings for Fashion Compatibility
• VITON: An Image-based Virtual Try-on Network
• Fine-grained Categorization and Dataset Bootstrapping using Deep Metric Learning with Humans in the Loop
• Learning Compatibility Across Categories for Heterogeneous Item Recommendation
• $c^{+}$GAN: Complementary Fashion Item Recommendation
• FashionNet: Personalized Outfit Recommendation with Deep Neural Network
• Modeling Visual Compatibility through Hierarchical Mid-level Elements
• Coherent and Controllable Outfit Generation
• Hierarchical Fashion Graph Network for Personalized Outfit Recommendation
• Deep Style Match for Complementary Recommendation
• Context-Aware Visual Compatibility Prediction
• Exploring AI in Fashion: A Review of Aesthetics, Personalization, Virtual Try-On, and Forecasting
• Complete the Look: Scene-based Complementary Product Recommendation
• CRAFT: Complementary Recommendations Using Adversarial Feature Transformer
• TransNFCM: Translation-Based Neural Fashion Compatibility Modeling
• Who Leads the Clothing Fashion: Style, Color, or Texture? A Computational Study
• Learning Tuple Compatibility for Conditional OutfitRecommendation
• Brand > Logo: Visual Analysis of Fashion Brands
• Recommending Outfits from Personal Closet
• Imitation Learning for Fashion Style Based on Hierarchical Multimodal Representation
• Fine-Grained Fashion Similarity Learning by Attribute-Specific Embedding Network