What Does CNN Shift Invariance Look Like? A Visualization Study
arXiv:2011.04127 · doi:10.1007/978-3-030-68238-5_15
Abstract
Feature extraction with convolutional neural networks (CNNs) is a popular method to represent images for machine learning tasks. These representations seek to capture global image content, and ideally should be independent of geometric transformations. We focus on measuring and visualizing the shift invariance of extracted features from popular off-the-shelf CNN models. We present the results of three experiments comparing representations of millions of images with exhaustively shifted objects, examining both local invariance (within a few pixels) and global invariance (across the image frame). We conclude that features extracted from popular networks are not globally invariant, and that biases and artifacts exist within this variance. Additionally, we determine that anti-aliased models significantly improve local invariance but do not impact global invariance. Finally, we provide a code repository for experiment reproduction, as well as a website to interact with our results at https://jakehlee.github.io/visualize-invariance.
Presented at the 2020 ECCV Workshop on Real-World Computer Vision from Inputs with Limited Quality (RLQ-TOD 2020), Glasgow, Scotland