Generative AI May Reinforce Social Biases in Software Engineering Education

Abstract

Generative artificial intelligence (GenAI) is increasingly being deployed across a wide range of real-world applications. Without careful evaluation, reliance on these systems can have unintended consequences, such as reinforcement of stereotypes and amplification of social biases. Such risks are particularly important in educational settings, where early decisions can shape students' interests, opportunities, and career trajectories. In this paper, we investigate how GenAI use by software engineering instructors may inadvertently reinforce software-specific social biases. We focus on two representative tasks: team formation based on student profiles and the generation of visual content for educational materials. Our results reveal significant biases in both tasks. In team formation, factors such as gender and nationality affect role assignments (e.g., women are more likely to be assigned to front-end roles than equally qualified men). In the visual content generation task, models generally produce diverse and balanced representations when depicting groups of individuals. However, when generating images of a single person, the outputs are predominantly male and light-skinned. These findings highlight the persistence of demographic biases in generative models and underscore the need for domain-specific evaluation and mitigation strategies to support their responsible use in educational settings.