Classification accuracy as a proxy for two sample testing

Ilmun Kim, Aaditya Ramdas, Aarti Singh +1

When data analysts train a classifier and check if its accuracy is significantly different from chance, they are implicitly performing a two-sample test. We investigate the statist…