SoK: Machine Learning Governance

Abstract

The application of machine learning (ML) in computer systems introduces not only many benefits but also risks to society. In this paper, we develop the concept of ML governance to balance such benefits and risks, with the aim of achieving responsible applications of ML. Our approach first systematizes research towards ascertaining ownership of data and models, thus fostering a notion of identity specific to ML systems. Building on this foundation, we use identities to hold principals accountable for failures of ML systems through both attribution and auditing. To increase trust in ML systems, we then survey techniques for developing assurance, i.e., confidence that the system meets its security requirements and does not exhibit certain known failures. This leads us to highlight the need for techniques that allow a model owner to manage the life cycle of their system, e.g., to patch or retire their ML system. Put altogether, our systematization of knowledge standardizes the interactions between principals involved in the deployment of ML throughout its life cycle. We highlight opportunities for future work, e.g., to formalize the resulting game between ML principals.

References in corpus (85)

• Deep Learning with Differential Privacy
• Language Models are Few-Shot Learners
• Self-Attention Generative Adversarial Networks
• Equality of Opportunity in Supervised Learning
• Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks
• Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
• Renyi Differential Privacy
• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks
• Certified Adversarial Robustness via Randomized Smoothing
• Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
• Federated Optimization:Distributed Optimization Beyond the Datacenter
• Certified Defenses against Adversarial Examples
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• BAE: BERT-based Adversarial Examples for Text Classification
• Generating Natural Adversarial Examples
• Exponentially Weighted Moving Average Charts for Detecting Concept Drift
• Deep Anomaly Detection with Outlier Exposure
• On the (Statistical) Detection of Adversarial Examples
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Learning Confidence for Out-of-Distribution Detection in Neural Networks
• Predictive Uncertainty Estimation via Prior Networks
• Scalable Private Learning with PATE
• Learning how to explain neural networks: PatternNet and PatternAttribution
• Identification of Deep Network Generated Images Using Disparities in Color Components
• Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
• Inverting Gradients -- How easy is it to break privacy in federated learning?
• Practical Secure Aggregation for Federated Learning on User-Held Data
• On the Convergence and Robustness of Adversarial Training
• Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition
• Robust Audio Adversarial Example for a Physical Attack
• Input complexity and out-of-distribution detection with likelihood-based generative models
• On Adaptive Attacks to Adversarial Example Defenses
• Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
• White-box vs Black-box: Bayes Optimal Strategies for Membership Inference
• Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
• Stolen Memories: Leveraging Model Memorization for Calibrated White-Box Membership Inference
• The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards
• A Nutritional Label for Rankings
• Thieves on Sesame Street! Model Extraction of BERT-based APIs
• Smooth Adversarial Training
• Recent Advances in Algorithmic High-Dimensional Robust Statistics
• SafetyNets: Verifiable Execution of Deep Neural Networks on an Untrusted Cloud
• Analyzing Information Leakage of Updates to Natural Language Models
• HotFlip: White-Box Adversarial Examples for Text Classification
• Auditing Differentially Private Machine Learning: How Private is Private SGD?
• On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping
• Detecting GAN-generated Imagery using Color Cues
• Remember What You Want to Forget: Algorithms for Machine Unlearning
• Differentially Private Learning Needs Better Features (or Much More Data)
• Aequitas: A Bias and Fairness Audit Toolkit
• Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks
• RAB: Provable Robustness Against Backdoor Attacks
• Adaptive Machine Unlearning
• Understanding and Improving Fast Adversarial Training
• Detecting and Simulating Artifacts in GAN Fake Images
• Towards Probabilistic Verification of Machine Unlearning
• Probably Approximately Metric-Fair Learning
• Encode, Shuffle, Analyze Privacy Revisited: Formalizations and Empirical Evaluation
• Fairer and more accurate, but for whom?
• Generalized No Free Lunch Theorem for Adversarial Robustness
• Dataset Inference: Ownership Resolution in Machine Learning
• Practical and Private (Deep) Learning without Sampling or Shuffling
• Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices
• Certified Data Removal from Machine Learning Models
• Intriguing properties of adversarial training at scale
• Performative Prediction
• Deep Partition Aggregation: Provable Defense against General Poisoning Attacks
• MT-Adapted Datasheets for Datasets: Template and Repository
• Analysis of Confident-Classifiers for Out-of-distribution Detection
• An Ethical Highlighter for People-Centric Dataset Creation
• Analyzing and Improving Neural Networks by Generating Semantic Counterexamples through Differentiable Rendering
• Solving Structured Hierarchical Games Using Differential Backward Induction
• Manipulating SGD with Data Ordering Attacks
• On the Privacy Risks of Algorithmic Fairness
• Secure Medical Image Analysis with CrypTFlow
• On Attribution of Deepfakes
• A General Framework For Detecting Anomalous Inputs to DNN Classifiers
• Proof-of-Learning: Definitions and Practice
• Do GANs leave artificial fingerprints?
• On the Exploitability of Audio Machine Learning Pipelines to Surreptitious Adversarial Examples
• Faking Fairness via Stealthily Biased Sampling

Cited by in corpus (2)

• On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning
• ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks