Interactive Interior Design Recommendation via Coarse-to-fine Multimodal Reinforcement Learning

Abstract

Personalized interior decoration design often incurs high labor costs. Recent efforts in developing intelligent interior design systems have focused on generating textual requirement-based decoration designs while neglecting the problem of how to mine homeowner's hidden preferences and choose the proper initial design. To fill this gap, we propose an Interactive Interior Design Recommendation System (IIDRS) based on reinforcement learning (RL). IIDRS aims to find an ideal plan by interacting with the user, who provides feedback on the gap between the recommended plan and their ideal one. To improve decision-making efficiency and effectiveness in large decoration spaces, we propose a Decoration Recommendation Coarse-to-Fine Policy Network (DecorRCFN). Additionally, to enhance generalization in online scenarios, we propose an object-aware feedback generation method that augments model training with diversified and dynamic textual feedback. Extensive experiments on a real-world dataset demonstrate our method outperforms traditional methods by a large margin in terms of recommendation accuracy. Further user studies demonstrate that our method reaches higher real-world user satisfaction than baseline methods.

arXiv commentAccepted by ACM International Conference on Multimedia'23. 9 pages, 7 figures

References in corpus (11)

• Zero-Shot Text-to-Image Generation
• OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework
• Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems
• Interactive Path Reasoning on Graph for Conversational Recommendation
• Composite Correlation Quantization for Efficient Multimodal Retrieval
• Dialog-based Interactive Image Retrieval
• ChatPainter: Improving Text to Image Generation using Dialogue
• Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese
• Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access
• From Show to Tell: A Survey on Deep Learning-based Image Captioning
• DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis

Cited by in corpus (1)

• EchoLadder: Progressive AI-Assisted Design of Immersive VR Scenes