Misty: UI Prototyping Through Interactive Conceptual Blending

Abstract

UI prototyping often involves iterating and blending elements from examples such as screenshots and sketches, but current tools offer limited support for incorporating these examples. Inspired by the cognitive process of conceptual blending, we introduce a novel UI workflow that allows developers to rapidly incorporate diverse aspects from design examples into work-in-progress UIs. We prototyped this workflow as Misty. Through an exploratory first-use study with 14 frontend developers, we assessed Misty's effectiveness and gathered feedback on this workflow. Our findings suggest that Misty's conceptual blending workflow helps developers kickstart creative explorations, flexibly specify intent in different stages of prototyping, and inspires developers through serendipitous UI blends. Misty demonstrates the potential for tools that blur the boundaries between developers and designers.

References in corpus (14)

• The What-If Tool: Interactive Probing of Machine Learning Models
• Homogenization Effects of Large Language Models on Human Creative Ideation
• Deep Learning for Source Code Modeling and Generation: Models, Applications and Challenges
• DirectGPT: A Direct Manipulation Interface to Interact with Large Language Models
• PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions
• Walking the Walk of AI Ethics: Organizational Challenges and the Individualization of Risk among Ethics Entrepreneurs
• How Culture Shapes What People Want From AI
• Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts
• Scout: Rapid Exploration of Interface Layout Alternatives through High-Level Design Constraints
• Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
• GANSpiration: Balancing Targeted and Serendipitous Inspiration in User Interface Design with Style-Based Generative Adversarial Network
• Auto-Icon+: An Automated End-to-End Code Generation Tool for Icon Designs in UI Development
• Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values
• DesignChecker: Visual Design Support for Blind and Low Vision Web Developers