Sequential Gallery for Interactive Visual Design Optimization

Abstract

Visual design tasks often involve tuning many design parameters. For example, color grading of a photograph involves many parameters, some of which non-expert users might be unfamiliar with. We propose a novel user-in-the-loop optimization method that allows users to efficiently find an appropriate parameter set by exploring such a high-dimensional design space through much easier two-dimensional search subtasks. This method, called sequential plane search, is based on Bayesian optimization to keep necessary queries to users as few as possible. To help users respond to plane-search queries, we also propose using a gallery-based interface that provides options in the two-dimensional subspace arranged in an adaptive grid view. We call this interactive framework Sequential Gallery since users sequentially select the best option from the options provided by the interface. Our experiment with synthetic functions shows that our sequential plane search can find satisfactory solutions in fewer iterations than baselines. We also conducted a preliminary user study, results of which suggest that novices can effectively complete search tasks with Sequential Gallery in a photo-enhancement scenario.

To be published at ACM Trans. Graph. (Proc. SIGGRAPH 2020); Project page available at https://koyama.xyz/project/sequential_gallery/

References in corpus (3)

• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
• Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design
• Towards the Automatic Anime Characters Creation with Generative Adversarial Networks

Cited by in corpus (21)

• PromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions
• Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques
• Active Learning and Bayesian Optimization: a Unified Perspective to Learn with a Goal
• Creating Emordle: Animating Word Cloud for Emotion Expression
• MemoVis: A GenAI-Powered Tool for Creating Companion Reference Images for 3D Design Feedback
• Continual Human-in-the-Loop Optimization
• The Impact of Expertise in the Loop for Exploring Machine Rationality
• Generative Melody Composition with Human-in-the-Loop Bayesian Optimization
• The Human in the Infinite Loop: A Case Study on Revealing and Explaining Human-AI Interaction Loop Failures
• Crowd-Powered Photo Enhancement Featuring an Active Learning Based Local Filter
• GenPara: Enhancing the 3D Design Editing Process by Inferring Users' Regions of Interest with Text-Conditional Shape Parameters
• Improving External Communication of Automated Vehicles Using Bayesian Optimization
• FontCraft: Multimodal Font Design Using Interactive Bayesian Optimization
• OptiCarVis: Improving Automated Vehicle Functionality Visualizations Using Bayesian Optimization to Enhance User Experience
• Computational Workflows for Designing Input Devices
• SwipeGANSpace: Swipe-to-Compare Image Generation via Efficient Latent Space Exploration
• Physically Consistent Preferential Bayesian Optimization for Food Arrangement
• Cooperative Design Optimization through Natural Language Interaction
• Discovering Diverse Athletic Jumping Strategies
• Efficient Hyperparameter Optimization for Physics-based Character Animation
• Tool- and Domain-Agnostic Parameterization of Style Transfer Effects Leveraging Pretrained Perceptual Metrics