Convolutions on Spherical Images

Abstract

Applying convolutional neural networks to spherical images requires particular considerations. We look to the millennia of work on cartographic map projections to provide the tools to define an optimal representation of spherical images for the convolution operation. We propose a representation for deep spherical image inference based on the icosahedral Snyder equal-area (ISEA) projection, a projection onto a geodesic grid, and show that it vastly exceeds the state-of-the-art for convolution on spherical images, improving semantic segmentation results by 12.6%.

Oral presentation at the 2019 SUMO Workshop on 360° Indoor Scene Understanding and Modeling at CVPR2019, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2019