4 papers
JudgmentBench: Comparing Rubric and Preference Evaluation for Quality Assessment
Russell Yang, Ruishi Chen, Pierce Kelaita +6
Two methodologies dominate current practices of benchmarking: rubric-based scoring evaluates items against predefined criteria, whereas comparative judgment elicits pairwise prefer…
Scalable Data Attribution via Forward-Only Test-Time Inference
Sibo Ma, Julian Nyarko
Data attribution seeks to trace model behavior back to the training examples that shaped it, enabling debugging, auditing, and data valuation at scale. Classical influence-function…
Identifying Emerging Concepts in Large Corpora
Sibo Ma, Julian Nyarko
We introduce a new method to identify emerging concepts in large text corpora. By analyzing changes in the heatmaps of the underlying embedding space, we are able to detect these c…
Breaking Down Bias: On The Limits of Generalizable Pruning Strategies
Sibo Ma, Alejandro Salinas, Peter Henderson +1
We employ model pruning to examine how LLMs conceptualize racial biases, and whether a generalizable mitigation strategy for such biases appears feasible. Our analysis yields sever…