4 papers · 1 filter
The Missing Evaluation Axis: What 10,000 Student Submissions Reveal About AI Tutor Effectiveness
Rose Niousha, Samantha Boatright Smith, Bita Akram +5
Current Artificial Intelligence (AI)-based tutoring systems (AI tutors) are primarily evaluated based on the pedagogical quality of their feedback messages. While important, pedago…
61A Bot Report: AI Assistants in CS1 Save Students Homework Time and Reduce Demands on Staff. (Now What?)
J. D. Zamfirescu-Pereira, Laryn Qi, Björn Hartmann +2
LLM-based chatbots enable students to get immediate, interactive help on homework assignments, but even a thoughtfully-designed bot may not serve all pedagogical goals. We report h…
Pensieve Discuss: Scalable Small-Group CS Tutoring System with AI
Yoonseok Yang, Jack Liu, J. D. Zamfirescu-Pereira +1
Small-group tutoring in Computer Science (CS) is effective, but presents the challenge of providing a dedicated tutor for each group and encouraging collaboration among group membe…
A Knowledge-Component-Based Methodology for Evaluating AI Assistants
Laryn Qi, J. D. Zamfirescu-Pereira, Taehan Kim +3
We evaluate an automatic hint generator for CS1 programming assignments powered by GPT-4, a large language model. This system provides natural language guidance about how students…