2 citations · 3 across the 8 of their papers we have counts for
8 papers
One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows
Zhuochun Li, Youngmin Ko, Ali Keramati +12
Recent agent benchmarks increasingly ground evaluation in executable environments, from code repair to web navigation, app APIs, and function calling. Yet completing consequential…
Simulating Students' Java Programming Errors with Large Language Models
Ali Keramati, Jie Cao, Iman Mohammadi +2
Understanding student errors in the programming is a cornerstone of programming education, yet obtaining a representative set of student errors for any newly designed task remains…
The Order Matters: Sequential Fine-Tuning of LLaMA for Coherent Automated Essay Scoring
Ali Keramati, Mark Warschauer
Automated Essay Scoring (AES) systems must judge interdependent discourse elements (e.g., lead, claim, evidence, conclusion), yet most approaches treat these in isolation, harming…
Early-Token Confidence Predicts Reasoning Quality in Multi-Agent LLM Debate
Ali Keramati, Justin Cheok, Jacob Horne +1
Evaluating reasoning quality in multi-agent LLM systems is challenging, especially for open-ended tasks without reference answers. We investigate whether intrinsic confidence signa…
The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge
Ali Keramati, Justin Cheok, Jacob Horne +1
Multi-agent debate systems are typically evaluated only on whether the final answer is correct, overlooking the quality of the intermediate reasoning that debate is designed to pro…
MADRAG: Multi-Agent Debate with Retrieval-Augmented Generation for Training-Free Analytic Essay Scoring
Ali Keramati, Shiyuan Zhou, Sharad Mehrotra +1
We present MADRAG, a training-free framework for analytic essay scoring that combines multi-agent reasoning with retrieval-augmented grounding. Unlike standard LLM-as-judge approac…
Disagreement as Data: Reasoning Trace Analytics in Multi-Agent Systems
Elham Tajik, Conrad Borchers, Bahar Shahrokhian +4
Learning analytics researchers often analyze qualitative student data such as coded annotations or interview transcripts to understand learning processes. With the rise of generati…
AI-powered Digital Framework for Personalized Economical Quality Learning at Scale
Mrzieh VatandoustMohammadieh, Mohammad Mahdi Mohajeri, Ali Keramati +1
The disparity in access to quality education is significant, both between developed and developing countries and within nations, regardless of their economic status. Socioeconomic…