Characterising Bias in Compressed Models

Abstract

The popularity and widespread use of pruning and quantization is driven by the severe resource constraints of deploying deep neural networks to environments with strict latency, memory and energy requirements. These techniques achieve high levels of compression with negligible impact on top-line metrics (top-1 and top-5 accuracy). However, overall accuracy hides disproportionately high errors on a small subset of examples; we call this subset Compression Identified Exemplars (CIE). We further establish that for CIE examples, compression amplifies existing algorithmic bias. Pruning disproportionately impacts performance on underrepresented features, which often coincides with considerations of fairness. Given that CIE is a relatively small subset but a great contributor of error in the model, we propose its use as a human-in-the-loop auditing tool to surface a tractable subset of the dataset for further inspection or annotation by a domain expert. We provide qualitative and quantitative support that CIE surfaces the most challenging examples in the data distribution for human-in-the-loop auditing.

References in corpus (15)

• Deep Residual Learning for Image Recognition
• Equality of Opportunity in Supervised Learning
• Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes
• The State of Sparsity in Deep Neural Networks
• No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World
• Exploring Sparsity in Recurrent Neural Networks
• MLIR: A Compiler Infrastructure for the End of Moore's Law
• Rigging the Lottery: Making All Tickets Winners
• Does Object Recognition Work for Everyone?
• Sparse DNNs with Improved Adversarial Robustness
• What Do Compressed Deep Neural Networks Forget?
• What is the State of Neural Network Pruning?
• Metric Learning for Novelty and Anomaly Detection
• Towards Compact and Robust Deep Neural Networks
• ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases

Cited by in corpus (25)

• On the Opportunities and Risks of Foundation Models
• Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
• A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
• Randomness In Neural Network Training: Characterizing The Impact of Tooling
• Deep Learning Through the Lens of Example Difficulty
• Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?
• Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models
• The Rich Get Richer: Disparate Impact of Semi-Supervised Learning
• Data-Efficient Pretraining via Contrastive Self-Supervision
• Who's responsible? Jointly quantifying the contribution of the learning algorithm and training data
• Beyond Fairness Metrics: Roadblocks and Challenges for Ethical AI in Practice
• NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM
• Generative Zero-shot Network Quantization
• Robustness Challenges in Model Distillation and Pruning for Natural Language Understanding
• Simon Says: Evaluating and Mitigating Bias in Pruned Neural Networks with Knowledge Distillation
• Scaling Up Exact Neural Network Compression by ReLU Stability
• DP-REC: Private & Communication-Efficient Federated Learning
• Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression
• Going Beyond Classification Accuracy Metrics in Model Compression
• Transitioning from Real to Synthetic data: Quantifying the bias in model
• Measure Twice, Cut Once: Quantifying Bias and Fairness in Deep Neural Networks
• Masked Training of Neural Networks with Partial Gradients
• Demystifying and Generalizing BinaryConnect
• The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation
• Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning