paper

Identifying Breakdowns in Conversational Recommender Systems using User Simulation

arXiv:2405.14249 · doi:10.1145/3640794.3665539

Abstract

We present a methodology to systematically test conversational recommender systems with regards to conversational breakdowns. It involves examining conversations generated between the system and simulated users for a set of pre-defined breakdown types, extracting responsible conversational paths, and characterizing them in terms of the underlying dialogue intents. User simulation offers the advantages of simplicity, cost-effectiveness, and time efficiency for obtaining conversations where potential breakdowns can be identified. The proposed methodology can be used as diagnostic tool as well as a development tool to improve conversational recommendation systems. We apply our methodology in a case study with an existing conversational recommender system and user simulator, demonstrating that with just a few iterations, we can make the system more robust to conversational breakdowns.

ACM Conversational User Interfaces 2024 (CUI '24), July 8--10, 2024, Luxembourg, Luxembourg

References in corpus (4)