ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems

Abstract

We present ConvLab-2, an open-source toolkit that enables researchers to build task-oriented dialogue systems with state-of-the-art models, perform an end-to-end evaluation, and diagnose the weakness of systems. As the successor of ConvLab (Lee et al., 2019b), ConvLab-2 inherits ConvLab's framework but integrates more powerful dialogue models and supports more datasets. Besides, we have developed an analysis tool and an interactive tool to assist researchers in diagnosing dialogue systems. The analysis tool presents rich statistics and summarizes common mistakes from simulated dialogues, which facilitates error analysis and system improvement. The interactive tool provides a user interface that allows developers to diagnose an assembled dialogue system by interacting with the system and modifying the output of each system component.

Accepted by ACL 2020 demo track

References in corpus (3)

• Why We Need New Evaluation Metrics for NLG
• Rasa: Open Source Language Understanding and Dialogue Management
• Plato Dialogue System: A Flexible Conversational AI Research Platform

Cited by in corpus (8)

• SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching
• Recent Advances and Challenges in Task-oriented Dialog System
• Robust Conversational AI with Grounded Text Generation
• AuGPT: Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models
• A Hybrid Task-Oriented Dialog System with Domain and Task Adaptive Pretraining
• Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog System
• How to Evaluate Your Dialogue Models: A Review of Approaches
• Integrating Pretrained Language Model for Dialogue Policy Learning