AI Alignment vs. AI Ethical Treatment: 10 Challenges

Abstract

A morally acceptable course of AI development should avoid two dangers: creating unaligned AI systems that pose a threat to humanity and mistreating AI systems that merit moral consideration in their own right. This paper argues these two dangers interact and that if we create AI systems that merit moral consideration, simultaneously avoiding both of these dangers would be extremely challenging. While our argument is straightforward and supported by a wide range of pretheoretical moral judgments, it has far-reaching moral implications for AI development. Although the most obvious way to avoid the tension between alignment and ethical treatment would be to avoid creating AI systems that merit moral consideration, this option may be unrealistic and is perhaps fleeting. So, we conclude by offering some suggestions for other ways of mitigating mistreatment risks associated with alignment.

author order is arbitrary

References in corpus (19)

• Consciousness in Artificial Intelligence: Insights from the Science of Consciousness
• Full-density multi-scale account of structure and dynamics of macaque visual cortex
• Could a Large Language Model be Conscious?
• Scalable agent alignment via reward modeling: a research direction
• Recursively Summarizing Books with Human Feedback
• Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
• Towards Evaluating AI Systems for Moral Status Using Self-Reports
• AI Research Considerations for Human Existential Safety (ARCHES)
• Supervising strong learners by amplifying weak experts
• On the link between conscious function and general intelligence in humans and machines
• Natural Selection Favors AIs over Humans
• Foundational Challenges in Assuring Alignment and Safety of Large Language Models
• Evaluating Language-Model Agents on Realistic Autonomous Tasks
• Digital Twin Brain: a simulation and assimilation platform for whole human brain
• Scaling Instructable Agents Across Many Simulated Worlds
• Scheming AIs: Will AIs fake alignment during training in order to get power?
• Steering Language Models With Activation Engineering
• Circuit Breaking: Removing Model Behaviors with Targeted Ablation
• AI Control: Improving Safety Despite Intentional Subversion