Insights from Social Shaping Theory: The Appropriation of Large Language Models in an Undergraduate Programming Course

Abstract

The capability of large language models (LLMs) to generate, debug, and explain code has sparked the interest of researchers and educators in undergraduate programming, with many anticipating their transformative potential in programming education. However, decisions about why and how to use LLMs in programming education may involve more than just the assessment of an LLM's technical capabilities. Using the social shaping of technology theory as a guiding framework, our study explores how students' social perceptions influence their own LLM usage. We then examine the correlation of self-reported LLM usage with students' self-efficacy and midterm performances in an undergraduate programming course. Triangulating data from an anonymous end-of-course student survey (n = 158), a mid-course self-efficacy survey (n=158), student interviews (n = 10), self-reported LLM usage on homework, and midterm performances, we discovered that students' use of LLMs was associated with their expectations for their future careers and their perceptions of peer usage. Additionally, early self-reported LLM usage in our context correlated with lower self-efficacy and lower midterm scores, while students' perceived over-reliance on LLMs, rather than their usage itself, correlated with decreased self-efficacy later in the course.

Accepted to the ACM Conference on International Computing Education Research V.1 (ICER '24 Vol. 1)

References in corpus (9)

• The Robots are Here: Navigating the Generative AI Revolution in Computing Education
• Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming
• CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs
• "It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers
• Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses
• CodeTailor: LLM-Powered Personalized Parsons Puzzles for Engaging Support While Learning Programming
• Exploring How Multiple Levels of GPT-Generated Programming Hints Support or Disappoint Novices
• Understanding the Effects of Using Parsons Problems to Scaffold Code Writing for Students with Varying CS Self-Efficacy Levels
• Integrating Personalized Parsons Problems with Multi-Level Textual Explanations to Scaffold Code Writing

Cited by in corpus (5)

• AI Literacy in K-12 and Higher Education in the Wake of Generative AI: An Integrative Review
• The Evolving Usage of GenAI by Computing Students
• How Scientists Use Large Language Models to Program
• Did Alice Do Wrong? Cross-Cultural Differences in Student Perceptions of Generative AI Use in University Computing Education
• Self-Regulated Personal Contracts as a Harm Reduction Approach to Generative AI in Undergraduate Programming Education