Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models

Abstract

Deep learning (DL) can achieve impressive results across a wide variety of tasks, but this often comes at the cost of training models for extensive periods on specialized hardware accelerators. This energy-intensive workload has seen immense growth in recent years. Machine learning (ML) may become a significant contributor to climate change if this exponential trend continues. If practitioners are aware of their energy and carbon footprint, then they may actively take steps to reduce it whenever possible. In this work, we present Carbontracker, a tool for tracking and predicting the energy and carbon footprint of training DL models. We propose that energy and carbon footprint of model development and training is reported alongside performance metrics using tools like Carbontracker. We hope this will promote responsible computing in ML and encourage research into energy-efficient deep neural networks.

arXiv commentAccepted to be presented at the ICML Workshop on "Challenges in Deploying and monitoring Machine Learning Systems", 2020. Source code at this link https://github.com/lfwa/carbontracker/

References in corpus (8)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Practical Bayesian Optimization of Machine Learning Algorithms
• Language Models are Few-Shot Learners
• Agent57: Outperforming the Atari Human Benchmark
• Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning
• Quantifying the Carbon Emissions of Machine Learning
• Lung Segmentation from Chest X-rays using Variational Data Imputation

Cited by in corpus (18)

• On the Opportunities and Risks of Foundation Models
• A first look into the carbon footprint of federated learning
• Towards Green Automated Machine Learning: Status Quo and Future Directions
• Lung Segmentation from Chest X-rays using Variational Data Imputation
• Keys to Accurate Feature Extraction Using Residual Spiking Neural Networks
• Dissecting FLOPs along input dimensions for GreenAI cost estimations
• EC-NAS: Energy Consumption Aware Tabular Benchmarks for Neural Architecture Search
• Greening AI-enabled Systems with Software Engineering: A Research Agenda for Environmentally Sustainable AI Practices
• InferBench: Understanding Deep Learning Inference Serving with an Automatic Benchmarking System
• Activation Compression of Graph Neural Networks using Block-wise Quantization with Improved Variance Minimization
• Distributed Deep Learning in Open Collaborations
• INFLECT-DGNN: Influencer Prediction with Dynamic Graph Neural Networks
• Uncertainty quantification in medical image segmentation with normalizing flows
• A Bregman Learning Framework for Sparse Neural Networks
• A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learning
• DeepSim: Semantic similarity metrics for learned image registration
• Multi-layered tensor networks for image classification
• Can Federated Learning Save The Planet?