One Perturbation Is Not Enough: Identifiability and Blind Baselines for Behavioral AI Evaluation

Rasul Khanbayov, Mariam Sohail, Ahmed Abdala +1

Behavioral evaluations perturb an input and read the induced change in the output in order to certify that a system uses that input. We show that the number of perturbations such a…