On Evaluating and Improving Conversational Agents in Production

Kasra Hosseini, Wen-Sen Cheng, Marco-Andrea Buchmann +2

We present a framework for evaluating and improving a large-scale, multi-agent shopping assistant in production, and report lessons from its use. Offline evaluation of such a syste…