Creating and Evaluating Personas Using Generative AI: A Scoping Review of 81 Articles

Abstract

As generative AI (GenAI) is increasingly applied in persona development to represent real users, understanding the implications and limitations of this technology is essential for establishing robust practices. This scoping review analyzes how 81 articles (2022-2025) use GenAI techniques for the creation, evaluation, and application of personas. The articles exhibited good level of reproducibility, with 61% of articles sharing resources (personas, code, or datasets). Furthermore, conversational persona interfaces are increasingly provided alongside traditional profiles. However, nearly half (45%) of the articles lack evaluation, and the majority (86%) use only GPT models. In some articles, GenAI use creates a risk of circularity, in which the same GenAI model both generates and evaluates outputs. Our findings also suggest that GenAI seems to reduce the role of human developers in the persona-creation process. To mitigate the associated risks, we propose actionable guidelines for the responsible integration of GenAI into persona development.

The previous article was updated to add more data

References in corpus (25)

• Out of One, Many: Using Language Models to Simulate Human Samples
• Writer-Defined AI Personas for On-Demand Feedback Generation
• Customizing Emotional Support: How Do Individuals Construct and Interact With LLM-Powered Chatbots
• An Empathy-Based Sandbox Approach to Bridge the Privacy Gap among Attitudes, Goals, Knowledge, and Behaviors
• PersonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation
• Risk or Chance? Large Language Models and Reproducibility in HCI Research
• Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs
• RecUserSim: A Realistic and Diverse User Simulator for Evaluating Conversational Recommender Systems
• Using Large Language Models to Create AI Personas for Replication, Generalization and Prediction of Media Effects: An Empirical Test of 133 Published Experimental Research Findings
• Writing user personas with Large Language Models: Testing phase 6 of a Thematic Analysis of semi-structured interviews
• Evaluation of LLMs Biases Towards Elite Universities: A Persona-Based Exploration
• Improved prompting and process for writing user personas with LLMs, using qualitative interviews: Capturing behaviour and personality traits of users
• Applying Psychometrics to Large Language Model Simulated Populations: Recreating the HEXACO Personality Inventory Experiment with Generative Agents
• Is persona enough for personality? Using ChatGPT to reconstruct an agent's latent personality from simple descriptions
• Co-persona: Leveraging LLMs and Expert Collaboration to Understand User Personas through Social Media Data Analysis
• Building Scaffolding Dialogue Data with LLM-Simulated Novices
• Agentic Society: Merging skeleton from real world and texture from Large Language Model
• SOMONITOR: Combining Explainable AI & Large Language Models for Marketing Analytics
• Exploring Augmentation and Cognitive Strategies for AI based Synthetic Personae
• SimuPanel: A Novel Immersive Multi-Agent System to Simulate Interactive Expert Panel Discussion
• "A Woman is More Culturally Knowledgeable than A Man?": The Effect of Personas on Cultural Norm Interpretation in LLMs
• Refusal as Silence: Gendered Disparities in Vision-Language Model Responses
• ChatChecker: A Framework for Dialogue System Testing and Evaluation Through Non-cooperative User Simulation
• LLMs' ways of seeing User Personas
• A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personas