"Is a picture of a bird a bird": Policy recommendations for dealing with ambiguity in machine vision models

Alicia Parrish, Sarah Laszlo, Lora Aroyo

Many questions that we ask about the world do not have a single clear answer, yet typical human annotation set-ups in machine learning assume there must be a single ground truth la…