Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning

Abstract

This paper strives to predict fine-grained fashion similarity. In this similarity paradigm, one should pay more attention to the similarity in terms of a specific design/attribute between fashion items. For example, whether the collar designs of the two clothes are similar. It has potential value in many fashion related applications, such as fashion copyright protection. To this end, we propose an Attribute-Specific Embedding Network (ASEN) to jointly learn multiple attribute-specific embeddings, thus measure the fine-grained similarity in the corresponding space. The proposed ASEN is comprised of a global branch and a local branch. The global branch takes the whole image as input to extract features from a global perspective, while the local branch takes as input the zoomed-in region-of-interest (RoI) w.r.t. the specified attribute thus able to extract more fine-grained features. As the global branch and the local branch extract the features from different perspectives, they are complementary to each other. Additionally, in each branch, two attention modules, i.e., Attribute-aware Spatial Attention and Attribute-aware Channel Attention, are integrated to make ASEN be able to locate the related regions and capture the essential patterns under the guidance of the specified attribute, thus make the learned attribute-specific embeddings better reflect the fine-grained similarity. Extensive experiments on three fashion-related datasets, i.e., FashionAI, DARN, and DeepFashion, show the effectiveness of ASEN for fine-grained fashion similarity prediction and its potential for fashion reranking. Code and data are available at https://github.com/maryeon/asenpp .

arXiv commentConference paper: arXiv:2002.02814

References in corpus (21)

• Adam: A Method for Stochastic Optimization
• Neural Machine Translation by Jointly Learning to Align and Translate
• Distilling the Knowledge in a Neural Network
• FaceNet: A Unified Embedding for Face Recognition and Clustering
• Squeeze-and-Excitation Networks
• NAIS: Neural Attentive Item Similarity Model for Recommendation
• Object-Part Attention Model for Fine-grained Image Classification
• Learning Fashion Compatibility with Bidirectional LSTMs
• Dual Encoding for Video Retrieval by Text
• Mining Fashion Outfit Composition Using An End-to-End Deep Learning Approach on Set Data
• Fine-graind Image Classification via Combining Vision and Language
• Dialog-based Interactive Image Retrieval
• Deep Learning for Fine-Grained Image Analysis: A Survey
• A Hierarchical Attention Model for Social Contextual Image Recommendation
• Fine-grained Visual-textual Representation Learning
• Cross-Domain Image Retrieval with Attention Modeling
• Deep Learning based Large Scale Visual Recommendation and Search for E-Commerce
• Exploring Human-like Attention Supervision in Visual Question Answering
• Who, Where, and What to Wear? Extracting Fashion Knowledge from Social Media
• Feature Re-Learning with Data Augmentation for Video Relevance Prediction
• On the Exploration of Incremental Learning for Fine-grained Image Retrieval

Cited by in corpus (3)

• From Region to Patch: Attribute-Aware Foreground-Background Contrastive Learning for Fine-Grained Fashion Retrieval
• MMFL-Net: Multi-scale and Multi-granularity Feature Learning for Cross-domain Fashion Retrieval
• Beyond Patches: Superpixel Token-based Transformers for Attribute-Specific Fashion Retrieval