Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts

Abstract

Creative coding tasks are often exploratory in nature. When producing digital artwork, artists usually begin with a high-level semantic construct such as a "stained glass filter" and programmatically implement it by varying code parameters such as shape, color, lines, and opacity to produce visually appealing results. Based on interviews with artists, it can be effortful to translate semantic constructs to program syntax, and current programming tools don't lend well to rapid creative exploration. To address these challenges, we introduce Spellburst, a large language model (LLM) powered creative-coding environment. Spellburst provides (1) a node-based interface that allows artists to create generative art and explore variations through branching and merging operations, (2) expressive prompt-based interactions to engage in semantic programming, and (3) dynamic prompt-driven interfaces and direct code editing to seamlessly switch between semantic and syntactic exploration. Our evaluation with artists demonstrates Spellburst's potential to enhance creative coding practices and inform the design of computational creativity tools that bridge semantic and syntactic spaces.

Cited by in corpus (17)

• DirectGPT: A Direct Manipulation Interface to Interact with Large Language Models
• A Taxonomy for Human-LLM Interaction Modes: An Initial Exploration
• WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualization
• KNOWNET: Guided Health Information Seeking from LLMs via Knowledge Graph Integration
• Beyond Code Generation: LLM-supported Exploration of the Program Design Space
• Patchview: LLM-Powered Worldbuilding with Generative Dust and Magnet Visualization
• AI-Assisted Causal Pathway Diagram for Human-Centered Design
• Misty: UI Prototyping Through Interactive Conceptual Blending
• Beyond Productivity: Rethinking the Impact of Creativity Support Tools
• SHAPE-IT: Exploring Text-to-Shape-Display for Generative Shape-Changing Behaviors with LLMs
• Human-AI Experience in Integrated Development Environments: A Systematic Literature Review
• JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models
• Advancing GUI for Generative AI: Charting the Design Space of Human-AI Interactions through Task Creativity and Complexity
• Composable Prompting Workspaces for Creative Writing: Exploration and Iteration Using Dynamic Widgets
• GestureCoach: Rehearsing for Engaging Talks with LLM-Driven Gesture Recommendations
• PaperBridge: Crafting Research Narratives through Human-AI Co-Exploration
• KinemaFX: A Kinematic-Driven Interactive System for Particle Effect Exploration and Customization