Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Images

Abstract

Distinguishing subtle differences in attributes is valuable, yet learning to make visual comparisons remains non-trivial. Not only is the number of possible comparisons quadratic in the number of training images, but also access to images adequately spanning the space of fine-grained visual differences is limited. We propose to overcome the sparsity of supervision problem via synthetically generated images. Building on a state-of-the-art image generation engine, we sample pairs of training images exhibiting slight modifications of individual attributes. Augmenting real training image pairs with these examples, we then train attribute ranking models to predict the relative strength of an attribute in novel pairs of real images. Our results on datasets of faces and fashion images show the great promise of bootstrapping imperfect image generators to counteract sample sparsity for learning to rank.

References in corpus (7)

• Auto-Encoding Variational Bayes
• DRAW: A Recurrent Neural Network For Image Generation
• Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition
• Deep Convolutional Inverse Graphics Network
• Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Images
• WhittleSearch: Interactive Image Search with Relative Attribute Feedback
• Convolutional Network for Attribute-driven and Identity-preserving Human Face Generation

Cited by in corpus (35)

• Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Images
• Free-form tumor synthesis in computed tomography images via richer generative adversarial network
• Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation
• Learning Type-Aware Embeddings for Fashion Compatibility
• A Broader Study of Cross-Domain Few-Shot Learning
• Learning to Sketch with Shortcut Cycle Consistency
• Image Morphing with Perceptual Constraints and STN Alignment
• Deception for Cyber Defence: Challenges and Opportunities
• Task-Aware Feature Generation for Zero-Shot Compositional Learning
• Learning Similarity Conditions Without Explicit Supervision
• DUAL-GLOW: Conditional Flow-Based Generative Model for Modality Transfer
• LaSO: Label-Set Operations networks for multi-label few-shot learning
• Reward Constrained Interactive Recommendation with Natural Language Feedback
• Fashionpedia: Ontology, Segmentation, and an Attribute Localization Dataset
• SyncGAN: Synchronize the Latent Space of Cross-modal Generative Adversarial Networks
• How to Not Measure Disentanglement
• Preference Modeling with Context-Dependent Salient Features
• Adversarial Semantic Data Augmentation for Human Pose Estimation
• Improved Confidence Bounds for the Linear Logistic Model and Applications to Linear Bandits
• Thinking Outside the Pool: Active Training Image Creation for Relative Attributes
• Interpreting the Latent Space of GANs via Correlation Analysis for Controllable Concept Manipulation
• Large-scale Benchmarks for Multimodal Recommendation with Ducho
• Recognizing Unseen States of Unknown Objects by Leveraging Knowledge Graphs
• Relative Attribute Classification with Deep Rank SVM
• Learning Nearest Neighbor Graphs from Noisy Distance Samples
• Deep Image Synthesis from Intuitive User Input: A Review and Perspectives
• Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback
• Tell-the-difference: Fine-grained Visual Descriptor via a Discriminating Referee
• A Modulation Module for Multi-task Learning with Applications in Image Retrieval
• Zero-Shot Compositional Concept Learning
• Learning Single/Multi-Attribute of Object with Symmetry and Group
• Compare and Contrast: Learning Prominent Visual Differences
• Cross-Domain Latent Modulation for Variational Transfer Learning
• Understanding Image Quality and Trust in Peer-to-Peer Marketplaces
• Domain Adaptation for Learning Generator from Paired Few-Shot Data