Trivial or impossible -- dichotomous data difficulty masks model differences (on ImageNet and beyond)

Kristof Meding, Luca M. Schulze Buschoff, Robert Geirhos +1

"The power of a generalization system follows directly from its biases" (Mitchell 1980). Today, CNNs are incredibly powerful generalisation systems -- but to what degree have we un…