Learning Data-Driven Objectives to Optimize Interactive Systems

Abstract

Effective optimization is essential for interactive systems to provide a satisfactory user experience. However, it is often challenging to find an objective to optimize for. Generally, such objectives are manually crafted and rarely capture complex user needs in an accurate manner. We propose an approach that infers the objective directly from observed user interactions. These inferences can be made regardless of prior knowledge and across different types of user behavior. We introduce interactive system optimization, a novel algorithm that uses these inferred objectives for optimization. Our main contribution is a new general principled approach to optimizing interactive systems using data-driven objectives. We demonstrate the high effectiveness of interactive system optimization over several simulations.

arXiv comment10 pages

References in corpus (9)

• Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Deep Reinforcement Learning for Dialogue Generation
• Learning to reinforcement learn
• Maximum Entropy Deep Inverse Reinforcement Learning
• One-Shot Visual Imitation Learning via Meta-Learning
• A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
• Towards Learning Reward Functions from User Interactions
• Correlation and Prediction of Evaluation Metrics in Information Retrieval