Sustainable AI: Environmental Implications, Challenges and Opportunities
arXiv:2111.00364
Abstract
This paper explores the environmental impact of the super-linear growth trends for AI from a holistic perspective, spanning Data, Algorithms, and System Hardware. We characterize the carbon footprint of AI computing by examining the model development cycle across industry-scale machine learning use cases and, at the same time, considering the life cycle of system hardware. Taking a step further, we capture the operational and manufacturing carbon footprint of AI computing and present an end-to-end analysis for what and how hardware-software design and at-scale optimization can help reduce the overall carbon footprint of AI. Based on the industry experience and lessons learned, we share the key challenges and chart out important development directions across the many dimensions of AI. We hope the key messages and insights presented in this paper can inspire the community to advance the field of AI in an environmentally-responsible manner.
References in corpus (27)
- Language Models are Few-Shot Learners
- On the Opportunities and Risks of Foundation Models
- Towards Federated Learning at Scale: System Design
- Deep Learning Recommendation Model for Personalization and Recommendation Systems
- Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
- Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning
- Quantifying the Carbon Emissions of Machine Learning
- Carbon Emissions and Large Neural Network Training
- Measuring the Algorithmic Efficiency of Neural Networks
- Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems
- Silent Data Corruptions at Scale
- ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning
- A first look into the carbon footprint of federated learning
- Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective
- Energy Usage Reports: Environmental awareness as part of algorithmic accountability
- Neural Architecture Search without Training
- TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning
- An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storage
- Check-N-Run: A Checkpointing System for Training Deep Learning Recommendation Models
- Salus: Fine-Grained GPU Sharing Primitives for Deep Learning Applications
- Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization
- Socio-Technological Challenges and Opportunities: Paths Forward
- CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery
- MLPerf Mobile Inference Benchmark
- Learning to Embed Categorical Features without Embedding Tables for Recommendation
- SVP-CF: Selection via Proxy for Collaborative Filtering Data
- Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative
Cited by in corpus (20)
- Carbon Explorer: A Holistic Approach for Designing Carbon Aware Datacenters
- How to estimate carbon footprint when training deep learning models? A guide and review
- Data-Centric Green AI: An Exploratory Empirical Study
- Neuromorphic Intermediate Representation: A Unified Instruction Set for Interoperable Brain-Inspired Computing
- Toward Sustainable HPC: Carbon Footprint Estimation and Environmental Implications of HPC Systems
- Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service
- CASPER: Carbon-Aware Scheduling and Provisioning for Distributed Web Services
- FedZero: Leveraging Renewable Excess Energy in Federated Learning
- A Synthesis of Green Architectural Tactics for ML-Enabled Systems
- A Survey on AI-driven Energy Optimisation in Terrestrial Next Generation Radio Access Networks
- Energy and Carbon Considerations of Fine-Tuning BERT
- Efficiency is Not Enough: A Critical Perspective of Environmentally Sustainable AI
- On the Promise and Pitfalls of Optimizing Embodied Carbon
- Efficient Self-Supervision using Patch-based Contrastive Learning for Histopathology Image Segmentation
- A Green(er) World for A.I
- MoralBERT: A Fine-Tuned Language Model for Capturing Moral Values in Social Discussions
- Materiality and Risk in the Age of Pervasive AI Sensors
- Identifying architectural design decisions for achieving green ML serving
- EnergAt: Fine-Grained Energy Attribution for Multi-Tenancy
- Towards Sustainable Artificial Intelligence: A Comprehensive Review and Comparative Analysis of Deep Learning Models' Carbon Footprint