CoKe: Localized Contrastive Learning for Robust Keypoint Detection
arXiv:2009.14115
Abstract
In this paper, we introduce a contrastive learning framework for keypoint detection (CoKe). Keypoint detection differs from other visual tasks where contrastive learning has been applied because the input is a set of images in which multiple keypoints are annotated. This requires the contrastive learning to be extended such that the keypoints are represented and detected independently, which enables the contrastive loss to make the keypoint features different from each other and from the background. Our approach has two benefits: It enables us to exploit contrastive learning for keypoint detection, and by detecting each keypoint independently the detection becomes more robust to occlusion compared to holistic methods, such as stacked hourglass networks, which attempt to detect all keypoints jointly. Our CoKe framework introduces several technical innovations. In particular, we introduce: (i) A clutter bank to represent non-keypoint features; (ii) a keypoint bank that stores prototypical representations of keypoints to approximate the contrastive loss between keypoints; and (iii) a cumulative moving average update to learn the keypoint prototypes while training the feature extractor. Our experiments on a range of diverse datasets (PASCAL3D+, MPII, ObjectNet3D) show that our approach works as well, or better than, alternative methods for keypoint detection, even for human keypoints, for which the literature is vast. Moreover, we observe that CoKe is exceptionally robust to partial occlusion and previously unseen object poses.
Accepted to WACV 2023
References in corpus (13)
- Distilling the Knowledge in a Neural Network
- A Simple Framework for Contrastive Learning of Visual Representations
- Learning deep representations by mutual information estimation and maximization
- Momentum Contrast for Unsupervised Visual Representation Learning
- Data-Efficient Image Recognition with Contrastive Predictive Coding
- Contrastive Multiview Coding
- Do Convnets Learn Correspondence?
- Supervised Contrastive Learning
- Multi-Context Attention for Human Pose Estimation
- Robustness of Object Recognition under Extreme Occlusion in Humans and Computational Models
- Dual Path Networks for Multi-Person Human Pose Estimation
- Detecting Semantic Parts on Partially Occluded Objects
- Robust Object Detection under Occlusion with Context-Aware CompositionalNets