5 papers
From Diagnosis to Redesign: Using Quantitative Ethnography to Improve Multi-Agent LLM Reasoning
Vedant Khatri, Anthony Cusimano, Zachari Swiecki +3
Multi-agent large language model (LLM) systems are designed to improve reasoning by decomposing tasks across multiple agents with specialized functions, but the presence of multipl…
Enhancing LLM-Based Data Annotation with Error Decomposition
Zhen Xu, Vedant Khatri, Yijun Dai +4
Large language models offer a scalable alternative to human coding for data annotation tasks, enabling the scale-up of research across data-intensive domains. While LLMs are alread…
Evaluating 21st-Century Competencies in Postsecondary Curricula with Large Language Models: Performance Benchmarking and Reasoning-Based Prompting Strategies
Zhen Xu, Xin Guan, Chenxi Shi +2
The growing emphasis on 21st-century competencies in postsecondary education, intensified by the transformative impact of generative AI, underscores the need to evaluate how these…
Bringing Pedagogy into Focus: Evaluating Virtual Teaching Assistants' Question-Answering in Asynchronous Learning Environments
Li Siyan, Zhen Xu, Vethavikashini Chithrra Raghuram +3
Asynchronous learning environments (ALEs) are widely adopted for formal and informal learning, but timely and personalized support is often limited. In this context, Virtual Teachi…
From Course to Skill: Evaluating LLM Performance in Curricular Analytics
Zhen Xu, Xinjin Li, Yingqi Huan +2
Curricular analytics (CA) -- systematic analysis of curricula data to inform program and course refinement -- becomes an increasingly valuable tool to help institutions align acade…