The Impact of AI on Developer Productivity: Evidence from GitHub Copilot

Abstract

Generative AI tools hold promise to increase human productivity. This paper presents results from a controlled experiment with GitHub Copilot, an AI pair programmer. Recruited software developers were asked to implement an HTTP server in JavaScript as quickly as possible. The treatment group, with access to the AI pair programmer, completed the task 55.8% faster than the control group. Observed heterogenous effects show promise for AI pair programmers to help people transition into software development careers.

Cited by in corpus (13)

• AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
• An Autoethnographic Case Study of Generative Artificial Intelligence's Utility for Accessibility
• ChatGPT in the classroom. Exploring its potential and limitations in a Functional Programming course
• Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chat
• Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow
• Large Language Models at Work in China's Labor Market
• Statically Contextualizing Large Language Models with Typed Holes
• NoCodeGPT: A No-Code Interface for Building Web Apps with Language Models
• Improving the Capabilities of Large Language Model Based Marketing Analytics Copilots With Semantic Search And Fine-Tuning
• OSINT Clinic: Co-designing AI-Augmented Collaborative OSINT Investigations for Vulnerability Assessment
• Synergizing LLMs and Knowledge Graphs: A Novel Approach to Software Repository-Related Question Answering
• Making AI Inevitable: Historical Perspective and the Problems of Predicting Long-Term Technological Change
• Developers' Experience with Generative AI -- First Insights from an Empirical Mixed-Methods Field Study