Semi-Supervised Visual Representation Learning for Fashion Compatibility
arXiv:2109.08052 · doi:10.1145/3460231.3474233
Abstract
We consider the problem of complementary fashion prediction. Existing approaches focus on learning an embedding space where fashion items from different categories that are visually compatible are closer to each other. However, creating such labeled outfits is intensive and also not feasible to generate all possible outfit combinations, especially with large fashion catalogs. In this work, we propose a semi-supervised learning approach where we leverage large unlabeled fashion corpus to create pseudo-positive and pseudo-negative outfits on the fly during training. For each labeled outfit in a training batch, we obtain a pseudo-outfit by matching each item in the labeled outfit with unlabeled items. Additionally, we introduce consistency regularization to ensure that representation of the original images and their transformations are consistent to implicitly incorporate colour and other important attributes through self-supervision. We conduct extensive experiments on Polyvore, Polyvore-D and our newly created large-scale Fashion Outfits datasets, and show that our approach with only a fraction of labeled examples performs on-par with completely supervised methods.
arXiv commentACM RecSys'21 (9 pages) DOI: https://dl.acm.org/doi/10.1145/3460231.3474233
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- Distributed Representations of Words and Phrases and their Compositionality
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- A Simple Framework for Contrastive Learning of Visual Representations
- Prototypical Networks for Few-shot Learning
- Improved Baselines with Momentum Contrastive Learning
- Unsupervised Representation Learning by Predicting Image Rotations
- Interpolation Consistency Training for Semi-Supervised Learning
- AutoAugment: Learning Augmentation Policies from Data
- CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features
- MixMatch: A Holistic Approach to Semi-Supervised Learning
- Deep Clustering for Unsupervised Learning of Visual Features
- Towards Inheritable Models for Open-Set Domain Adaptation
- Context-Aware Visual Compatibility Prediction
- Self-supervised Visual Attribute Learning for Fashion Compatibility