456 citations · 984 across the 34 of their papers we have counts for
8 papers · 1 filter
Identifying Incorrect Annotations in Multi-Label Classification Data
Aditya Thyagarajan, Elías Snorrason, Curtis Northcutt +1
In multi-label classification, each example in a dataset may be annotated as belonging to one or more classes (or none of the classes). Example applications include image (or docum…
Detecting Label Errors in Token Classification Data
Wei-Chen Wang, Jonas Mueller
Mislabeled examples are a common issue in real-world data, particularly for tasks like token classification where many labels must be chosen on a fine-grained basis. Here we consid…
CROWDLAB: Supervised learning to infer consensus labels and quality scores for data with multiple annotators
Hui Wen Goh, Ulyana Tkachenko, Jonas Mueller
Real-world data for classification is often labeled by multiple annotators. For analyzing such data, we introduce CROWDLAB, a straightforward approach to utilize any trained classi…
Time-Varying Propensity Score to Bridge the Gap between the Past and Present
Rasool Fakoor, Jonas Mueller, Zachary C. Lipton +2
Real-world deployment of machine learning models is challenging because data evolves over time. While no model can work when data evolves in an arbitrary fashion, if there is some…
Back to the Basics: Revisiting Out-of-Distribution Detection Baselines
Johnson Kuan, Jonas Mueller
We study simple methods for out-of-distribution (OOD) image detection that are compatible with any already trained classifier, relying on only its predictions or learned representa…
DataPerf: Benchmarks for Data-Centric AI Development
Mark Mazumder, Colby Banbury, Xiaozhe Yao +42
Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithf…