Unpacking the Expressed Consequences of AI Research in Broader Impact Statements

Abstract

The computer science research community and the broader public have become increasingly aware of negative consequences of algorithmic systems. In response, the top-tier Neural Information Processing Systems (NeurIPS) conference for machine learning and artificial intelligence research required that authors include a statement of broader impact to reflect on potential positive and negative consequences of their work. We present the results of a qualitative thematic analysis of a sample of statements written for the 2020 conference. The themes we identify broadly fall into categories related to how consequences are expressed (e.g., valence, specificity, uncertainty), areas of impacts expressed (e.g., bias, the environment, labor, privacy), and researchers' recommendations for mitigating negative consequences in the future. In light of our results, we offer perspectives on how the broader impact statement can be implemented in future iterations to better align with potential goals.

12 pages, AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) 2021; added reference and link to dataset

References in corpus (16)

• Institutionalising Ethics in AI through Broader Impact Requirements
• Unsupervised Learning of Dense Visual Representations
• Against Scale: Provocations and Resistances to Scale Thinking
• Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms
• Adapting to Misspecification in Contextual Bandits
• Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks
• Collapsing Bandits and Their Application to Public Health Interventions
• Like a Researcher Stating Broader Impact For the Very First Time
• ColdGANs: Taming Language GANs with Cautious Sampling Strategies
• Stateful Posted Pricing with Vanishing Regret via Dynamic Deterministic Markov Decision Processes
• 3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data
• Profile Entropy: A Fundamental Measure for the Learnability and Compressibility of Discrete Distributions
• Optimal Query Complexity of Secure Stochastic Convex Optimization
• Preference learning along multiple criteria: A game-theoretic perspective
• Revisiting the Sample Complexity of Sparse Spectrum Approximation of Gaussian Processes
• Breaking Speech Recognizers to Imagine Lyrics

Cited by in corpus (12)

• How Different Groups Prioritize Ethical Values for Responsible AI
• Farsight: Fostering Responsible AI Awareness During AI Application Prototyping
• Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Survey
• Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooM
• "That's important, but...": How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations
• The Ethical Implications of Generative Audio Models: A Systematic Literature Review
• REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research
• ESR: Ethics and Society Review of Artificial Intelligence Research
• Crowdsourcing Impacts: Exploring the Utility of Crowds for Anticipating Societal Impacts of Algorithmic Decision Making
• Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work
• AI Ethics Statements -- Analysis and lessons learnt from NeurIPS Broader Impact Statements
• Addressing Privacy Threats from Machine Learning