Harms from Increasingly Agentic Algorithmic Systems

Abstract

Research in Fairness, Accountability, Transparency, and Ethics (FATE) has established many sources and forms of algorithmic harm, in domains as diverse as health care, finance, policing, and recommendations. Much work remains to be done to mitigate the serious harms of these systems, particularly those disproportionately affecting marginalized communities. Despite these ongoing harms, new systems are being developed and deployed which threaten the perpetuation of the same harms and the creation of novel ones. In response, the FATE community has emphasized the importance of anticipating harms. Our work focuses on the anticipation of harms from increasingly agentic systems. Rather than providing a definition of agency as a binary property, we identify 4 key characteristics which, particularly in combination, tend to increase the agency of a given algorithmic system: underspecification, directness of impact, goal-directedness, and long-term planning. We also discuss important harms which arise from increasing agency -- notably, these include systemic and/or long-range impacts, often on marginalized stakeholders. We emphasize that recognizing agency of algorithmic systems does not absolve or shift the human responsibility for algorithmic harms. Rather, we use the term agency to highlight the increasingly evident fact that ML systems are not fully under human control. Our work explores increasingly agentic algorithmic systems in three parts. First, we explain the notion of an increase in agency for algorithmic systems in the context of diverse perspectives on agency across disciplines. Second, we argue for the need to anticipate harms from increasingly agentic systems. Third, we discuss important harms from increasingly agentic systems and ways forward for addressing them. We conclude by reflecting on implications of our work for anticipating algorithmic harms from emerging systems.

arXiv commentAccepted at FAccT 2023

References in corpus (55)

• Survey of Hallucination in Natural Language Generation
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
• On the Opportunities and Risks of Foundation Models
• Model Cards for Model Reporting
• Scaling Laws for Neural Language Models
• Flamingo: a Visual Language Model for Few-Shot Learning
• Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
• Underspecification Presents Challenges for Credibility in Modern Machine Learning
• A Survey on In-context Learning
• Scaling Language Models: Methods, Analysis & Insights from Training Gopher
• The Fallacy of AI Functionality
• Measurement and Fairness
• Roles for Computing in Social Change
• Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
• Inner Monologue: Embodied Reasoning through Planning with Language Models
• Predictability and Surprise in Large Generative Models
• Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language
• Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learning
• Degenerate Feedback Loops in Recommender Systems
• Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning
• The Forgotten Margins of AI Ethics
• Mastering Diverse Domains through World Models
• The Grey Hoodie Project: Big Tobacco, Big Tech, and the threat on academic integrity
• Horizon: Facebook's Open Source Applied Reinforcement Learning Platform
• Towards Accountability in the Use of Artificial Intelligence for Public Administrations
• Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders
• Meaning without reference in large language models
• Teaching language models to support answers with verified quotes
• Discovering Language Model Behaviors with Model-Written Evaluations
• PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change
• Data Governance in the Age of Large-Scale Data-Driven Language Technology
• The Algorithmic Imprint
• Scaling Laws for Reward Model Overoptimization
• Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark
• Transparency, Governance and Regulation of Algorithmic Tools Deployed in the Criminal Justice System: a UK Case Study
• Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
• Human-Timescale Adaptation in an Open-Ended Task Space
• Fairness in Recommendation: Foundations, Methods and Applications
• Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
• Broken Neural Scaling Laws
• Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks
• What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring
• Language Models as Agent Models
• Reinforcement learning based recommender systems: A survey
• Legitimacy, Authority, and Democratic Duties of Explanation
• AI Ethics for Systemic Issues: A Structural Approach
• Agents and Devices: A Relative Definition of Agency
• Discovering Agents
• The Alberta Plan for AI Research
• Reward Reports for Reinforcement Learning
• Scaling laws for single-agent reinforcement learning
• Path-Specific Objectives for Safer Agent Incentives
• The AI Economist: Optimal Economic Policy Design via Two-level Deep Reinforcement Learning

Cited by in corpus (9)

• AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
• Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directions
• The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships
• Black-Box Access is Insufficient for Rigorous AI Audits
• Frontier AI developers need an internal audit function
• The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems
• Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security
• Exploring Culturally Informed AI Assistants: A Comparative Study of ChatBlackGPT and ChatGPT
• From Model Performance to Claim: How a Change of Focus in Machine Learning Replicability Can Help Bridge the Responsibility Gap