Accuracy-Efficiency Trade-Offs and Accountability in Distributed ML Systems

Abstract

Trade-offs between accuracy and efficiency pervade law, public health, and other non-computing domains, which have developed policies to guide how to balance the two in conditions of uncertainty. While computer science also commonly studies accuracy-efficiency trade-offs, their policy implications remain poorly examined. Drawing on risk assessment practices in the US, we argue that, since examining these trade-offs has been useful for guiding governance in other domains, we need to similarly reckon with these trade-offs in governing computer systems. We focus our analysis on distributed machine learning systems. Understanding the policy implications in this area is particularly urgent because such systems, which include autonomous vehicles, tend to be high-stakes and safety-critical. We 1) describe how the trade-off takes shape for these systems, 2) highlight gaps between existing US risk assessment standards and what these systems require to be properly assessed, and 3) make specific calls to action to facilitate accountability when hypothetical risks concerning the accuracy-efficiency trade-off become realized as accidents in the real world. We close by discussing how such accountability mechanisms encourage more just, transparent governance aligned with public values.

References in corpus (9)

• Scaling Laws for Neural Language Models
• Deep Learning with Limited Numerical Precision
• Compressing Deep Convolutional Networks using Vector Quantization
• Provably Efficient Reinforcement Learning with Linear Function Approximation
• Performance-Efficiency Trade-off of Low-Precision Numerical Formats in Deep Neural Networks
• A Replication Study: Machine Learning Models Are Capable of Predicting Sexual Orientation From Facial Images
• Model Selection's Disparate Impact in Real-World Deep Learning Applications
• Asymptotically Optimal Exact Minibatch Metropolis-Hastings
• The Homeostasis Protocol: Avoiding Transaction Coordination Through Program Analysis

Cited by in corpus (4)

• Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning
• Four Years of FAccT: A Reflexive, Mixed-Methods Analysis of Research Contributions, Shortcomings, and Future Prospects
• Non-Determinism and the Lawlessness of Machine Learning Code
• Making the Unaccountable Internet: The Changing Meaning of Accounting in the Early ARPANET