Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences

Abstract

On-device machine learning (ML) promises to improve the privacy, responsiveness, and proliferation of new, intelligent user experiences by moving ML computation onto everyday personal devices. However, today's large ML models must be drastically compressed to run efficiently on-device, a hurtle that requires deep, yet currently niche expertise. To engage the broader human-centered ML community in on-device ML experiences, we present the results from an interview study with 30 experts at Apple that specialize in producing efficient models. We compile tacit knowledge that experts have developed through practical experience with model compression across different hardware platforms. Our findings offer pragmatic considerations missing from prior work, covering the design process, trade-offs, and technical strategies that go into creating efficient models. Finally, we distill design recommendations for tooling to help ease the difficulty of this work and bring on-device ML into to more widespread practice.

Proceedings of the 2024 ACM CHI Conference on Human Factors in Computing Systems

References in corpus (12)

• Improving fairness in machine learning systems: What do industry practitioners need?
• Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
• How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
• Enabling hand gesture customization on wrist-worn devices
• AutoDS: Towards Human-Centered Automation of Data Science
• Modeling Assumptions Clash with the Real World: Transparency, Equity, and Community Challenges for Student Assignment Algorithms
• Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development
• Exploring the Opportunities of AR for Enriching Storytelling with Family Photos between Grandparents and Grandchildren
• Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy
• Designing and Training of Lightweight Neural Networks on Edge Devices using Early Halting in Knowledge Distillation

Cited by in corpus (2)

• Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments
• Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference