Beyond Planar Symmetry: Modeling human perception of reflection and rotation symmetries in the wild

Abstract

Humans take advantage of real world symmetries for various tasks, yet capturing their superb symmetry perception mechanism with a computational model remains elusive. Motivated by a new study demonstrating the extremely high inter-person accuracy of human perceived symmetries in the wild, we have constructed the first deep-learning neural network for reflection and rotation symmetry detection (Sym-NET), trained on photos from MS-COCO (Microsoft-Common Object in COntext) dataset with nearly 11K consistent symmetry-labels from more than 400 human observers. We employ novel methods to convert discrete human labels into symmetry heatmaps, capture symmetry densely in an image and quantitatively evaluate Sym-NET against multiple existing computer vision algorithms. On CVPR 2013 symmetry competition testsets and unseen MS-COCO photos, Sym-NET significantly outperforms all other competitors. Beyond mathematically well-defined symmetries on a plane, Sym-NET demonstrates abilities to identify viewpoint-varied 3D symmetries, partially occluded symmetrical objects, and symmetries at a semantic level.

arXiv commentTo appear in the International Conference on Computer Vision (ICCV) 2017

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Residual Learning for Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Going Deeper with Convolutions
• Fully Convolutional Networks for Semantic Segmentation
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Convolutional Pose Machines
• Attention to Scale: Scale-aware Semantic Image Segmentation
• Flowing ConvNets for Human Pose Estimation in Videos
• Designing Deep Networks for Surface Normal Estimation
• Harmonic Networks: Deep Translation and Rotation Equivariance
• Object Skeleton Extraction in Natural Images by Fusing Scale-associated Deep Side Outputs