Chat-crowd: A Dialog-based Platform for Visual Layout Composition

Abstract

In this paper we introduce Chat-crowd, an interactive environment for visual layout composition via conversational interactions. Chat-crowd supports multiple agents with two conversational roles: agents who play the role of a designer are in charge of placing objects in an editable canvas according to instructions or commands issued by agents with a director role. The system can be integrated with crowdsourcing platforms for both synchronous and asynchronous data collection and is equipped with comprehensive quality controls on the performance of both types of agents. We expect that this system will be useful to build multimodal goal-oriented dialog tasks that require spatial and geometric reasoning.

References in corpus (4)

• Dialogue Act Modeling for Automatic Tagging and Recognition of Conversational Speech
• Exploring Models and Data for Image Question Answering
• Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation
• CoDraw: Collaborative Drawing as a Testbed for Grounded Goal-driven Communication