Toward Inclusive AI-Driven Development: Exploring Gender Differences in Code Generation Tool Interactions

Abstract

The increasing reliance on Code Generation Tools (CGTs), such as Claude Code and GitHub Copilot, is revamping programming workflows and raising critical questions about fairness and inclusivity in human-AI collaboration. While CGTs offer potential productivity enhancements, their effectiveness across diverse user groups have not been sufficiently investigated. We hypothesized that developers' interactions with CGTs vary based on gender, influencing task outcomes and cognitive load, as prior research suggests that gender differences can affect technology use and cognitive processing. This study employed a mixed-subjects design with 39 participants, evenly divided by gender for a counterbalanced design. Participants completed two programming tasks of medium to high difficulty using two distinct treatments: only CGT assistance and only internet access. Task orders and conditions were counterbalanced to mitigate order effects. We collected cognitive load surveys, screen recordings, and task performance metrics such as completion time, code correctness, and CGT interaction behaviors. Our results indicate no statistically significant gender differences in cognitive load or performance outcomes when using CGTs compared to Internet-based workflows. CGTs reduce intrinsic and extraneous cognitive load compared to Internet based workflows, but the reduction was not statistically significantly. However, CGTs improved advanced code correctness. Our results suggest that CGTs can lower cognitive load and enhance performance on complex coding tasks without significantly affecting core correctness or completion time. These findings highlight how CGT usage can reduce cognitive burden and support more equitable programming experiences across users.

Stage 2 RR under review at EMSE. The accepted Stage 1 protocol is publicly archived on OSF (DOI:https://doi.org/10.17605/OSF.IO/TCFJR)