79 citations · 125 across the 14 of their papers we have counts for
16 papers
Terminal-Bench-LILT: Multilingual Agentic Coding Benchmark Grounded in Language, Region, and Culture
Yunsu Kim, Kaden Uhlig, Ashwin Purohit +10
Most evaluations for coding agents are conducted exclusively in English, which does not reflect real-world multilingual deployment. We present Terminal-Bench-LILT, a suite of 300 a…
The Missing Evaluation Axis: What 10,000 Student Submissions Reveal About AI Tutor Effectiveness
Rose Niousha, Samantha Boatright Smith, Bita Akram +5
Current Artificial Intelligence (AI)-based tutoring systems (AI tutors) are primarily evaluated based on the pedagogical quality of their feedback messages. While important, pedago…
ParaStudent: Closing the Sim2Real Gap in User Simulators for AI Tutor Evaluation
Rose Niousha, Mihran Miroyan, Abigail O'Neill +4
Evaluating Artificial Intelligence (AI) tutor feedback before deployment requires anticipating student engagement, typically assessed through real interaction data. We introduce Pa…
Cross-lingual Human-Preference Alignment for Neural Machine Translation with Direct Quality Optimization
Kaden Uhlig, Joern Wuebker, Raphael Reinauer +1
Reinforcement Learning from Human Feedback (RLHF) and derivative techniques like Direct Preference Optimization (DPO) are task-alignment algorithms used to repurpose general, found…
Pensieve Discuss: Scalable Small-Group CS Tutoring System with AI
Yoonseok Yang, Jack Liu, J. D. Zamfirescu-Pereira +1
Small-group tutoring in Computer Science (CS) is effective, but presents the challenge of providing a dedicated tutor for each group and encouraging collaboration among group membe…
A Knowledge-Component-Based Methodology for Evaluating AI Assistants
Laryn Qi, J. D. Zamfirescu-Pereira, Taehan Kim +3
We evaluate an automatic hint generator for CS1 programming assignments powered by GPT-4, a large language model. This system provides natural language guidance about how students…