Beyond Top-Class Agreement: Using Divergences to Forecast Performance under Distribution Shift

Mona Schirmer, Dan Zhang, Eric Nalisnick

Knowing if a model will generalize to data 'in the wild' is crucial for safe deployment. To this end, we study model disagreement notions that consider the full predictive distribu…