Truthful AI: Developing and governing AI that does not lie
arXiv:2110.06674
Abstract
In many contexts, lying -- the use of verbal falsehoods to deceive -- is harmful. While lying has traditionally been a human affair, AI systems that make sophisticated verbal statements are becoming increasingly prevalent. This raises the question of how we should limit the harm caused by AI "lies" (i.e. falsehoods that are actively selected for). Human truthfulness is governed by social norms and by laws (against defamation, perjury, and fraud). Differences between AI and humans present an opportunity to have more precise standards of truthfulness for AI, and to have these standards rise over time. This could provide significant benefits to public epistemics and the economy, and mitigate risks of worst-case AI futures. Establishing norms or laws of AI truthfulness will require significant work to: (1) identify clear truthfulness standards; (2) create institutions that can judge adherence to those standards; and (3) develop AI systems that are robustly truthful. Our initial proposals for these areas include: (1) a standard of avoiding "negligent falsehoods" (a generalisation of lies that is easier to assess); (2) institutions to evaluate AI systems before and after real-world deployment; and (3) explicitly training AI systems to be truthful via curated datasets and human interaction. A concerning possibility is that evaluation mechanisms for eventual truthfulness standards could be captured by political interests, leading to harmful censorship and propaganda. Avoiding this might take careful attention. And since the scale of AI speech acts might grow dramatically over the coming decades, early truthfulness standards might be particularly important because of the precedents they set.
References in corpus (23)
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Language Models are Few-Shot Learners
- European Union regulations on algorithmic decision-making and a "right to explanation"
- Scaling Laws for Neural Language Models
- Deep reinforcement learning from human preferences
- The Pile: An 800GB Dataset of Diverse Text for Language Modeling
- Accurate Uncertainties for Deep Learning Using Calibrated Regression
- Measuring Massive Multitask Language Understanding
- Towards a Human-like Open-Domain Chatbot
- A Simple Language Model for Task-Oriented Dialogue
- Trial without Error: Towards Safe Reinforcement Learning via Human Intervention
- Finetuned Language Models Are Zero-Shot Learners
- Generative Language Modeling for Automated Theorem Proving
- Alignment of Language Agents
- Learning to summarize from human feedback
- Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets
- Emergent Multi-Agent Communication in the Deep Learning Era
- AI safety via debate
- AI Research Considerations for Human Existential Safety (ARCHES)
- Supervising strong learners by amplifying weak experts
- Risks from Learned Optimization in Advanced Machine Learning Systems
- CommonsenseQA 2.0: Exposing the Limits of AI through Gamification
- Robust Artificial Intelligence and Robust Human Organizations