325 citations · 400 across the 19 of their papers we have counts for
21 papers
Generative AI Produces and Transmits Beauty Bias
AJ Alvero, Deepak Varuvel Dennison, Alexandra Crespin +5
Societies reward people judged to be attractive by standards intertwined with gender and race. In this paper, we examine if and how generative AI may contribute to these dynamics.…
Sandpiper: Orchestrated AI-Annotation for Educational Discourse at Scale
Daryl Hedley, Doug Pietrzak, Jorge Dias +9
Digital educational environments are expanding toward complex AI and human discourse, providing researchers with an abundance of data that offers deep insights into learning and in…
LLM Reasoning Predicts When Models Are Right: Evidence from Coding Classroom Discourse
Bakhtawar Ahtisham, Kirk Vanacore, Zhuqian Zhou +2
Large Language Models (LLMs) are increasingly deployed to automatically label and analyze educational dialogue at scale, yet current pipelines lack reliable ways to detect when mod…
Utility-Preserving De-Identification for Math Tutoring: Investigating Numeric Ambiguity in the MathEd-PII Benchmark Dataset
Zhuqian Zhou, Kirk Vanacore, Bakhtawar Ahtisham +7
Large-scale sharing of dialogue data is key to advancing the science of teaching and learning, yet rigorous de-identification remains a major barrier. In mathematics tutoring trans…
Does Algorithmic Uncertainty Sway Human Experts? Evidence from a Field Experiment in Selective College Admissions
Hansol Lee, AJ Alvero, René F. Kizilcec +1
Algorithmic predictions are inherently uncertain: even models with similar aggregate accuracy can produce different predictions for the same individual, raising concerns that high-…
Codebook-Injected Dialogue Segmentation for Multi-Utterance Constructs Annotation: LLM-Assisted and Gold-Label-Free Evaluation
Jinsook Lee, Kirk Vanacore, Zhuqian Zhou +3
Dialogue Act (DA) annotation typically treats communicative or pedagogical intent as localized to individual utterances or turns. This leads annotators to agree on the underlying a…
AI Annotation Orchestration: Evaluating LLM verifiers to Improve the Quality of LLM Annotations in Learning Analytics
Bakhtawar Ahtisham, Kirk Vanacore, Jinsook Lee +3
Large Language Models (LLMs) are increasingly used to annotate learning interactions, yet concerns about reliability limit their utility. We test whether verification-oriented orch…
When the Past Misleads: Rethinking Training Data Expansion Under Temporal Distribution Shifts
Chengyuan Yao, Yunxuan Tang, Christopher Brooks +2
Predictive models are typically trained on historical data to predict future outcomes. While it is commonly assumed that training on more historical data would improve model perfor…
Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essays
Jinsook Lee, AJ Alvero, Thorsten Joachims +1
People are increasingly using technologies equipped with large language models (LLM) to write texts for formal communication, which raises two important questions at the intersecti…
Algorithms for College Admissions Decision Support: Impacts of Policy Change and Inherent Variability
Jinsook Lee, Emma Harvey, Joyce Zhou +3
Each year, selective American colleges sort through tens of thousands of applications to identify a first-year class that displays both academic merit and diversity. In the 2023-20…
The Life Cycle of Large Language Models: A Review of Biases in Education
Jinsook Lee, Yann Hicke, Renzhe Yu +2
Large Language Models (LLMs) are increasingly adopted in educational contexts to provide personalized support to students and teachers. The unprecedented capacity of LLM-based appl…
Fairness Hub Technical Briefs: Definition and Detection of Distribution Shift
Nicolas Acevedo, Carmen Cortez, Chris Brooks +2
Distribution shift is a common situation in machine learning tasks, where the data used for training a model is different from the data the model is applied to in the real world. T…
Cultural Differences in Students' Privacy Concerns in Learning Analytics across Germany, South Korea, Spain, Sweden, and the United States
Olga Viberg, René F. Kizilcec, Ioana Jivet +5
Applications of learning analytics (LA) can raise concerns from students about their privacy in higher education contexts. Developing effective privacy-enhancing practices requires…
What Explains Teachers' Trust of AI in Education across Six Countries?
Olga Viberg, Mutlu Cukurova, Yael Feldman-Maggor +9
With growing expectations to use AI-based educational technology (AI-EdTech) to improve students' learning outcomes and enrich teaching practice, teachers play a central role in th…
Cultural Bias and Cultural Alignment of Large Language Models
Yan Tao, Olga Viberg, Ryan S. Baker +1
Culture fundamentally shapes people's reasoning, behavior, and communication. As people increasingly use generative artificial intelligence (AI) to expedite and automate personal a…
Fairness Hub Technical Briefs: AUC Gap
Jinsook Lee, Chris Brooks, Renzhe Yu +1
To measure bias, we encourage teams to consider using AUC Gap: the absolute difference between the highest and lowest test AUC for subgroups (e.g., gender, race, SES, prior knowled…
Augmenting Holistic Review in University Admission using Natural Language Processing for Essays and Recommendation Letters
Jinsook Lee, Bradon Thymes, Joyce Zhou +2
University admission at many highly selective institutions uses a holistic review process, where all aspects of the application, including protected attributes (e.g., race, gender)…
Should College Dropout Prediction Models Include Protected Attributes?
Renzhe Yu, Hansol Lee, René F. Kizilcec
Early identification of college dropouts can provide tremendous value for improving student success and institutional effectiveness, and predictive analytics are increasingly used…
On the limits of algorithmic prediction across the globe
Xingyu Li, Difan Song, Miaozhe Han +2
The impact of predictive algorithms on people's lives and livelihoods has been noted in medicine, criminal justice, finance, hiring and admissions. Most of these algorithms are dev…
Algorithmic Fairness in Education
René F. Kizilcec, Hansol Lee
Data-driven predictive models are increasingly used in education to support students, instructors, and administrators. However, there are concerns about the fairness of the predict…
Evaluation of Fairness Trade-offs in Predicting Student Success
Hansol Lee, René F. Kizilcec
Predictive models for identifying at-risk students early can help teaching staff direct resources to better support them, but there is a growing concern about the fairness of algor…