Understanding Generalization via Leave-One-Out Conditional Mutual Information

Mahdi Haghifam, Shay Moran, Daniel M. Roy +1

We study the mutual information between (certain summaries of) the output of a learning algorithm and its $n$ training data, conditional on a supersample of $n+1$ i.i.d. data from…