5 papers
Agreement Is Not Quality: Blind Expert Verification of Human and LLM Qualitative Coding When Human Consensus Is Not Ground Truth
Alex Liu, Lief Esbenshade, Michael Xiao +4
Evaluations of LLM-assisted qualitative coding almost universally measure model performance as agreement with human coders, a practice that presumes human coding is the standard to…
Human-LLM Collaborative Inductive Coding for Conceptualizing K-12 Educator AI Use
Alex Liu, Min Sun, Lief Esbenshade +4
Qualitative researchers increasingly encounter interaction corpora whose scale exceeds what manual coding alone can address, and large language models (LLMs) are frequently propose…
PCS-UQ: Uncertainty Quantification via the Predictability-Computability-Stability Framework
Abhineet Agarwal, Fange Xiao, Rebecca Barter +3
As machine learning (ML) enters high-stakes domains, trustworthy uncertainty quantification (UQ) is essential for safety. In this paper we introduce PCS-UQ, a framework based on th…
Creating and Evaluating K-12 GenAI Assessment Graders Through Context Engineering
Zewei Tian, Alex Liu, Lief Esbenshade +6
The integration of large language models (LLMs) into educational assessment represents a transformative shift in classroom grading practices. While automated scoring systems and ma…
Generative AI in K-12 Classrooms: A Midyear Implementation Report
Lief Esbenshade, Alex Liu, Michael Xiao +6
This mid-year report summarizes teacher use of Colleague AI across 12 Washington State school districts from September 1 to December 31, 2025. Produced jointly by Colleague AI and…