CurlingNet: Compositional Learning between Images and Text for Fashion IQ Data

Abstract

We present an approach named CurlingNet that can measure the semantic distance of composition of image-text embedding. In order to learn an effective image-text composition for the data in the fashion domain, our model proposes two key components as follows. First, the Delivery makes the transition of a source image in an embedding space. Second, the Sweeping emphasizes query-related components of fashion images in the embedding space. We utilize a channel-wise gating mechanism to make it possible. Our single model outperforms previous state-of-the-art image-text composition models including TIRG and FiLM. We participate in the first fashion-IQ challenge in ICCV 2019, for which ensemble of our model achieves one of the best performances.

arXiv comment4 pages, 4 figures, ICCV 2019 Linguistics Meets image and video retrieval workshop, Fashion IQ challenge

References in corpus (7)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Distributed Representations of Words and Phrases and their Compositionality
• Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
• Learnable pooling with Context Gating for video classification
• Use What You Have: Video Retrieval Using Representations From Collaborative Experts
• Fashion-Gen: The Generative Fashion Dataset and Challenge
• Automatic Spatially-aware Fashion Concept Discovery

Cited by in corpus (4)

• Modality-Agnostic Attention Fusion for visual search with text feedback
• RTIC: Residual Learning for Text and Image Composition using Graph Convolutional Network
• Cycled Compositional Learning between Images and Text
• Conversational Fashion Image Retrieval via Multiturn Natural Language Feedback