6 citations · 10 across the 5 of their papers we have counts for
5 papers
Measurement to Meaning: A Validity-Centered Framework for AI Evaluation
Olawale Salaudeen, Anka Reuel, Ahmed Ahmed +6
While the capabilities and utility of AI systems have advanced, rigorous norms for evaluating these systems have lagged. Grand claims, such as models achieving general reasoning ca…
Quantifying the Importance of Data Alignment in Downstream Model Performance
Krrish Chawla, Aryan Sahai, Mario DePavia +4
Contrary to the conventional emphasis on dataset size, we explore the role of data alignment -- an often overlooked aspect of data quality -- in training capable Large Language Mod…
WONDERBREAD: A Benchmark for Evaluating Multimodal Foundation Models on Business Process Management Tasks
Michael Wornow, Avanika Narayan, Ben Viggiano +15
Existing ML benchmarks lack the depth and diversity of annotations needed for evaluating models on business process management (BPM) tasks. BPM is the practice of documenting, meas…
Random Serial Dictatorship with Transfers
Sudharsan Sundar, Eric Gao, Trevor Chow +1
It is well known that Random Serial Dictatorship is strategy-proof and leads to a Pareto-Efficient outcome. We show that this result breaks down when individuals are allowed to mak…
Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data
Brando Miranda, Alycia Lee, Sudharsan Sundar +4
Current trends in pre-training Large Language Models (LLMs) primarily focus on the scaling of model and dataset size. While the quality of pre-training data is considered an import…