FashionFail: Addressing Failure Cases in Fashion Object Detection and Segmentation

Abstract

In the realm of fashion object detection and segmentation for online shopping images, existing state-of-the-art fashion parsing models encounter limitations, particularly when exposed to non-model-worn apparel and close-up shots. To address these failures, we introduce FashionFail; a new fashion dataset with e-commerce images for object detection and segmentation. The dataset is efficiently curated using our novel annotation tool that leverages recent foundation models. The primary objective of FashionFail is to serve as a test bed for evaluating the robustness of models. Our analysis reveals the shortcomings of leading models, such as Attribute-Mask R-CNN and Fashionformer. Additionally, we propose a baseline approach using naive data augmentation to mitigate common failure cases and improve model robustness. Through this work, we aim to inspire and support further research in fashion item detection and segmentation for industrial applications. The dataset, annotation tool, code, and models are available at \url{https://rizavelioglu.github.io/fashionfail/}.

to be published in 2024 International Joint Conference on Neural Networks (IJCNN)

References in corpus (8)

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
• Visual Search at eBay
• Visual Search at Alibaba
• Benchmarking Detection Transfer Learning with Vision Transformers
• Shop The Look: Building a Large Scale Visual Shopping System at Pinterest
• Bootstrapping Complete The Look at Pinterest
• ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification