AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models
arXiv:2308.06507 · doi:10.18653/v1/2023.acl-short.149
Abstract
Information-seeking conversation, which aims to help users gather information through conversation, has achieved great progress in recent years. However, the research is still stymied by the scarcity of training data. To alleviate this problem, we propose AutoConv for synthetic conversation generation, which takes advantage of the few-shot learning ability and generation capacity of large language models (LLM). Specifically, we formulate the conversation generation problem as a language modeling task, then finetune an LLM with a few human conversations to capture the characteristics of the information-seeking process and use it for generating synthetic conversations with high quality. Experimental results on two frequently-used datasets verify that AutoConv has substantial improvements over strong baselines and alleviates the dependence on human annotation. In addition, we also provide several analysis studies to promote future research.
arXiv commentAccepted to ACL 2023 Main Conference (Short)
References in corpus (12)
- Adam: A Method for Stochastic Optimization
- Unsupervised Data Augmentation for Consistency Training
- The Curious Case of Neural Text Degeneration
- Multitask Prompted Training Enables Zero-Shot Task Generalization
- GLM-130B: An Open Bilingual Pre-trained Model
- Towards Understanding and Mitigating Social Biases in Language Models
- Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
- Recipes for Safety in Open-domain Chatbots
- A Contrastive Framework for Neural Text Generation
- UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training
- Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation
- Dialog Inpainting: Turning Documents into Dialogs