SPROUT: an Interactive Authoring Tool for Generating Programming Tutorials with the Visualization of Large Language Models

Abstract

The rapid development of large language models (LLMs), such as ChatGPT, has revolutionized the efficiency of creating programming tutorials. LLMs can be instructed with text prompts to generate comprehensive text descriptions of code snippets. However, the lack of transparency in the end-to-end generation process has hindered the understanding of model behavior and limited user control over the generated results. To tackle this challenge, we introduce a novel approach that breaks down the programming tutorial creation task into actionable steps. By employing the tree-of-thought method, LLMs engage in an exploratory process to generate diverse and faithful programming tutorials. We then present SPROUT, an authoring tool equipped with a series of interactive visualizations that empower users to have greater control and understanding of the programming tutorial creation process. A formal user study demonstrated the effectiveness of SPROUT, showing that our tool assists users to actively participate in the programming tutorial creation process, leading to more reliable and customizable results. By providing users with greater control and understanding, SPROUT enhances the user experience and improves the overall quality of programming tutorial. A free copy of this paper and all supplemental materials are available at https://osf.io/uez2t/?view_only=5102e958802341daa414707646428f86.

References in corpus (19)

• Evaluating Large Language Models Trained on Code
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• Automatic Generation of Programming Exercises and Code Explanations using Large Language Models
• Comparing Code Explanations Created by Students and Large Language Models
• MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
• Sensecape: Enabling Multilevel Exploration and Sensemaking with Large Language Models
• Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation
• ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing
• Graphologue: Exploring Large Language Model Responses with Interactive Diagrams
• PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation
• DirectGPT: A Direct Manipulation Interface to Interact with Large Language Models
• VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping
• Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture
• LaMPost: Design and Evaluation of an AI-assisted Email Writing Prototype for Adults with Dyslexia
• XNLI: Explaining and Diagnosing NLI-based Visual Data Analysis
• PromptAid: Prompt Exploration, Perturbation, Testing and Iteration using Visual Analytics for Large Language Models
• ITSS: Interactive Web-Based Authoring and Playback Integrated Environment for Programming Tutorials
• Exploring Large Language Models as a Source of Common-Sense Knowledge for Robots
• CoLadder: Supporting Programmers with Hierarchical Code Generation in Multi-Level Abstraction