A Note On Interpreting Canary Exposure

Abstract

Canary exposure, introduced in Carlini et al. is frequently used to empirically evaluate, or audit, the privacy of machine learning model training. The goal of this note is to provide some intuition on how to interpret canary exposure, including by relating it to membership inference attacks and differential privacy.

short note, edited to add a sentence on independence of canary losses, including adding Pillutla et al