Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling

Abstract

Over the last several years, end-to-end neural conversational agents have vastly improved in their ability to carry a chit-chat conversation with humans. However, these models are often trained on large datasets from the internet, and as a result, may learn undesirable behaviors from this data, such as toxic or otherwise harmful language. Researchers must thus wrestle with the issue of how and when to release these models. In this paper, we survey the problem landscape for safety for end-to-end conversational AI and discuss recent and related work. We highlight tensions between values, potential positive impact and potential harms, and provide a framework for making decisions about whether and how to release these models, following the tenets of value-sensitive design. We additionally provide a suite of tools to enable researchers to make better-informed decisions about training and releasing end-to-end conversational AI models.

References in corpus (30)

• Language Models are Few-Shot Learners
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• A Neural Conversational Model
• A Deep Reinforced Model for Abstractive Summarization
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Release Strategies and the Social Impacts of Language Models
• Towards a Human-like Open-Domain Chatbot
• Suicidal Ideation Detection: A Review of Machine Learning Methods and Applications
• Conversational AI: The Science Behind the Alexa Prize
• Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge
• Plug and Play Language Models: A Simple Approach to Controlled Text Generation
• DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
• Recipes for Safety in Open-domain Chatbots
• ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons
• Adversarial NLI: A New Benchmark for Natural Language Understanding
• Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize
• Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets
• SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
• HateCheck: Functional Tests for Hate Speech Detection Models
• Neural Generation Meets Real People: Towards Emotionally Engaging Mixed-Initiative Conversations
• Revealing Persona Biases in Dialogue Systems
• Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models
• Challenges in Automated Debiasing for Toxic Language Detection
• Does Gender Matter? Towards Fairness in Dialogue Systems
• Reducing malicious use of synthetic media research: Considerations and potential release practices for machine learning
• Detoxifying Language Models Risks Marginalizing Minority Voices
• Deploying Lifelong Open-Domain Dialogue Learning
• Detecting and Classifying Malevolent Dialogue Responses: Taxonomy, Data and Methodology
• Detect All Abuse! Toward Universal Abusive Language Detection Models

Cited by in corpus (5)

• On the Opportunities and Risks of Foundation Models
• Few-Shot Bot: Prompt-Based Learning for Dialogue Systems
• ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AI
• A Word on Machine Ethics: A Response to Jiang et al. (2021)
• Cheap Learning: Maximising Performance of Language Models for Social Data Science Using Minimal Data