44 citations · 70 across the 5 of their papers we have counts for
Showing cs.CLShow all
2 papers · 1 filter
cs.CL2025
PIE: Performance Interval Estimation for Free-Form Generation Tasks
Chi-Yang Hsu, Alexander Braylan, Yiheng Su +2
Confidence estimation infers a probability for whether each model output is correct or not. While predicting such binary correctness is sensible for tasks with exact answers, free-…
cs.CL2022★ 19 cited
Measuring Annotator Agreement Generally across Complex Structured, Multi-object, and Free-text Annotation Tasks
Alexander Braylan, Omar Alonso, Matthew Lease
When annotators label data, a key metric for quality assurance is inter-annotator agreement (IAA): the extent to which annotators agree on their labels. Though many IAA measures ex…