Accuracy-Efficiency Trade-Offs and Accountability in Distributed ML Systems

Abstract

Trade-offs between accuracy and efficiency pervade law, public health, and other non-computing domains, which have developed policies to guide how to balance the two in conditions of uncertainty. While computer science also commonly studies accuracy-efficiency trade-offs, their policy implications remain poorly examined. Drawing on risk assessment practices in the US, we argue that, since examining these trade-offs has been useful for guiding governance in other domains, we need to similarly reckon with these trade-offs in governing computer systems. We focus our analysis on distributed machine learning systems. Understanding the policy implications in this area is particularly urgent because such systems, which include autonomous vehicles, tend to be high-stakes and safety-critical. We 1) describe how the trade-off takes shape for these systems, 2) highlight gaps between existing US risk assessment standards and what these systems require to be properly assessed, and 3) make specific calls to action to facilitate accountability when hypothetical risks concerning the accuracy-efficiency trade-off become realized as accidents in the real world. We close by discussing how such accountability mechanisms encourage more just, transparent governance aligned with public values.

References in corpus (19)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• TensorFlow: A system for large-scale machine learning
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Scaling Laws for Neural Language Models
• HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
• Deep Learning with Limited Numerical Precision
• Compressing Deep Convolutional Networks using Vector Quantization
• QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
• Provably Efficient Reinforcement Learning with Linear Function Approximation
• Staleness-aware Async-SGD for Distributed Deep Learning
• High-Accuracy Low-Precision Training
• Performance-Efficiency Trade-off of Low-Precision Numerical Formats in Deep Neural Networks
• CYCLADES: Conflict-free Asynchronous Machine Learning
• Exploration of Low Numeric Precision Deep Learning Inference Using Intel FPGAs
• Ensuring Rapid Mixing and Low Bias for Asynchronous Gibbs Sampling
• A Replication Study: Machine Learning Models Are Capable of Predicting Sexual Orientation From Facial Images
• Model Selection's Disparate Impact in Real-World Deep Learning Applications
• The Homeostasis Protocol: Avoiding Transaction Coordination Through Program Analysis
• Asymptotically Optimal Exact Minibatch Metropolis-Hastings

Cited by in corpus (4)

• Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning
• Four Years of FAccT: A Reflexive, Mixed-Methods Analysis of Research Contributions, Shortcomings, and Future Prospects
• Non-Determinism and the Lawlessness of Machine Learning Code
• Making the Unaccountable Internet: The Changing Meaning of Accounting in the Early ARPANET