Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset

Abstract

Conversational search is an approach to information retrieval (IR), where users engage in a dialogue with an agent in order to satisfy their information needs. Previous conceptual work described properties and actions a good agent should exhibit. Unlike them, we present a novel conceptual model defined in terms of conversational goals, which enables us to reason about current research practices in conversational search. Based on the literature, we elicit how existing tasks and test collections from the fields of IR, natural language processing (NLP) and dialogue systems (DS) fit into this model. We describe a set of characteristics that an ideal conversational search dataset should have. Lastly, we introduce MANtIS (the code and dataset are available at https://guzpenha.github.io/MANtIS/), a large-scale dataset containing multi-domain and grounded information seeking dialogues that fulfill all of our dataset desiderata. We provide baseline results for the conversation response ranking and user intent prediction tasks.

References in corpus (5)

• Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems
• Simple Applications of BERT for Ad Hoc Document Retrieval
• Analyzing and Characterizing User Intent in Information-seeking Conversations
• Attentive Memory Networks: Efficient Machine Reading for Conversational Search
• A Hybrid Retrieval-Generation Neural Conversation Model

Cited by in corpus (7)

• QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
• Unstructured Text Enhanced Open-domain Dialogue System: A Systematic Survey
• Multi-Stage Conversational Passage Retrieval: An Approach to Fusing Term Importance Estimation and Neural Query Rewriting
• Conversations with Search Engines: SERP-based Conversational Response Generation
• doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset
• Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking
• On the Calibration and Uncertainty of Neural Learning to Rank Models