369 citations · 1.2k across the 41 of their papers we have counts for
16 papers · 1 filter
A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education
Jacob Doughty, Zipiao Wan, Anishka Bompelli +12
There is a constant need for educators to develop and maintain effective up-to-date assessments. While there is a growing body of research in computing education on utilizing large…
From GPT-3 to GPT-4: On the Evolving Efficacy of LLMs to Answer Multiple-choice Questions for Programming Classes in Higher Education
Jaromir Savelka, Arav Agarwal, Christopher Bogart +1
We explore the evolving efficacy of three generative pre-trained transformer (GPT) models in generating answers for multiple-choice questions (MCQ) from introductory and intermedia…
From Text to Structure: Using Large Language Models to Support the Development of Legal Expert Systems
Samyar Janatian, Hannes Westermann, Jinzhe Tan +2
Encoding legislative text in a formal representation is an important prerequisite to different tasks in the field of AI & Law. For example, rule-based expert systems focused on leg…
Efficient Classification of Student Help Requests in Programming Courses Using Large Language Models
Jaromir Savelka, Paul Denny, Mark Liffiton +1
The accurate classification of student help requests with respect to the type of help being sought can enable the tailoring of effective responses. Automatically classifying such r…
Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies
Jakub Drápal, Hannes Westermann, Jaromir Savelka
Thematic analysis and other variants of inductive coding are widely used qualitative analytic methods within empirical legal studies (ELS). We propose a novel framework facilitatin…
Patterns of Student Help-Seeking When Using a Large Language Model-Powered Programming Assistant
Brad Sheese, Mark Liffiton, Jaromir Savelka +1
Providing personalized assistance at scale is a long-standing challenge for computing educators, but a new generation of tools powered by large language models (LLMs) offers immens…
The Robots are Here: Navigating the Generative AI Revolution in Computing Education
James Prather, Paul Denny, Juho Leinonen +12
Recent advancements in artificial intelligence (AI) are fundamentally reshaping computing, with large language models (LLMs) now effectively being able to generate and interpret so…
CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes
Mark Liffiton, Brad Sheese, Jaromir Savelka +1
Computing educators face significant challenges in providing timely support to students, especially in large class settings. Large language models (LLMs) have emerged recently and…
LLMediator: GPT-4 Assisted Online Dispute Resolution
Hannes Westermann, Jaromir Savelka, Karim Benyekhlef
In this article, we introduce LLMediator, an experimental platform designed to enhance online dispute resolution (ODR) by utilizing capabilities of state-of-the-art large language…
Harnessing LLMs in Curricular Design: Using GPT-4 to Support Authoring of Learning Objectives
Pragnya Sridhar, Aidan Doyle, Arav Agarwal +3
We evaluated the capability of a generative pre-trained transformer (GPT-4) to automatically generate high-quality learning objectives (LOs) in the context of a practically oriente…
Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?
Jaromir Savelka, Kevin D. Ashley, Morgan A Gray +2
We evaluated the capability of generative pre-trained transformers~(GPT-4) in analysis of textual data in tasks that require highly specialized domain expertise. Specifically, we f…
Explaining Legal Concepts with Augmented Large Language Models (GPT-4)
Jaromir Savelka, Kevin D. Ashley, Morgan A. Gray +2
Interpreting the meaning of legal open-textured terms is a key task of legal professionals. An important source for this interpretation is how the term was applied in previous cour…
Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses
Jaromir Savelka, Arav Agarwal, Marshall An +2
This paper studies recent developments in large language models' (LLM) abilities to pass assessments in introductory and intermediate Python programming courses at the postsecondar…
Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Texts
Jaromir Savelka
We evaluated the capability of a state-of-the-art generative pre-trained transformer (GPT) model to perform semantic annotation of short text snippets (one to few sentences) coming…
Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?
Jaromir Savelka, Arav Agarwal, Christopher Bogart +2
We evaluated the capability of generative pre-trained transformers (GPT), to pass assessments in introductory and intermediate Python programming courses at the postsecondary level…
Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code
Jaromir Savelka, Arav Agarwal, Christopher Bogart +1
We analyzed effectiveness of three generative pre-trained transformer (GPT) models in answering multiple-choice question (MCQ) assessments, often involving short snippets of code,…