Distribution Estimation to Automate Transformation Policies for Self-Supervision

Abstract

In recent visual self-supervision works, an imitated classification objective, called pretext task, is established by assigning labels to transformed or augmented input images. The goal of pretext can be predicting what transformations are applied to the image. However, it is observed that image transformations already present in the dataset might be less effective in learning such self-supervised representations. Building on this observation, we propose a framework based on generative adversarial network to automatically find the transformations which are not present in the input dataset and thus effective for the self-supervised learning. This automated policy allows to estimate the transformation distribution of a dataset and also construct its complementary distribution from which training pairs are sampled for the pretext task. We evaluated our framework using several visual recognition datasets to show the efficacy of our automated transformation policy.

arXiv commentNeurIPS 2021 Workshop: Self-Supervised Learning - Theory and Practice

References in corpus (9)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)
• What Makes for Good Views for Contrastive Learning?
• Exploiting Unlabeled Data in CNNs by Self-supervised Learning to Rank
• Supervised Contrastive Learning
• Grasp2Vec: Learning Object Representations from Self-Supervised Grasping
• Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning
• Two Stream Self-Supervised Learning for Action Recognition
• Towards a Hypothesis on Visual Transformation based Self-Supervision