On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks

Stephen Mussmann, Robin Jia, Percy Liang

Many pairwise classification tasks, such as paraphrase detection and open-domain question answering, naturally have extreme label imbalance (e.g., $99.99\%$ of examples are negativ…