Alignment of Language Agents
arXiv:2103.14659
Abstract
For artificial intelligence to be beneficial to humans the behaviour of AI agents needs to be aligned with what humans want. In this paper we discuss some behavioural issues for language agents, arising from accidental misspecification by the system designer. We highlight some ways that misspecification can occur and discuss some behavioural issues that could arise from misspecification, including deceptive or manipulative language, and review some approaches for avoiding these issues.
References in corpus (13)
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Language Models are Few-Shot Learners
- Underspecification Presents Challenges for Credibility in Modern Machine Learning
- Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models
- Scalable agent alignment via reward modeling: a research direction
- AI Safety Gridworlds
- Language Understanding for Text-based Games Using Deep Reinforcement Learning
- Assessing Social and Intersectional Biases in Contextualized Word Representations
- AI safety via debate
- Supervising strong learners by amplifying weak experts
- Risks from Learned Optimization in Advanced Machine Learning Systems
- Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
- Understanding Agent Incentives using Causal Influence Diagrams. Part I: Single Action Settings
Cited by in corpus (8)
- On the Opportunities and Risks of Foundation Models
- Evaluating Large Language Models Trained on Code
- Unsolved Problems in ML Safety
- Deception Abilities Emerged in Large Language Models
- A Survey on Symbolic Knowledge Distillation of Large Language Models
- Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets
- Truthful AI: Developing and governing AI that does not lie
- Collaborative Storytelling with Human Actors and AI Narrators