Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models

Abstract

Deep learning (DL) can achieve impressive results across a wide variety of tasks, but this often comes at the cost of training models for extensive periods on specialized hardware accelerators. This energy-intensive workload has seen immense growth in recent years. Machine learning (ML) may become a significant contributor to climate change if this exponential trend continues. If practitioners are aware of their energy and carbon footprint, then they may actively take steps to reduce it whenever possible. In this work, we present Carbontracker, a tool for tracking and predicting the energy and carbon footprint of training DL models. We propose that energy and carbon footprint of model development and training is reported alongside performance metrics using tools like Carbontracker. We hope this will promote responsible computing in ML and encourage research into energy-efficient deep neural networks.

Accepted to be presented at the ICML Workshop on "Challenges in Deploying and monitoring Machine Learning Systems", 2020. Source code at this link https://github.com/lfwa/carbontracker/

References in corpus (5)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Practical Bayesian Optimization of Machine Learning Algorithms
• Language Models are Few-Shot Learners
• Quantifying the Carbon Emissions of Machine Learning
• Lung Segmentation from Chest X-rays using Variational Data Imputation

Cited by in corpus (10)

• Lung Segmentation from Chest X-rays using Variational Data Imputation
• EC-NAS: Energy Consumption Aware Tabular Benchmarks for Neural Architecture Search
• Greening AI-enabled Systems with Software Engineering: A Research Agenda for Environmentally Sustainable AI Practices
• InferBench: Understanding Deep Learning Inference Serving with an Automatic Benchmarking System
• Distributed Deep Learning in Open Collaborations
• Activation Compression of Graph Neural Networks using Block-wise Quantization with Improved Variance Minimization
• INFLECT-DGNN: Influencer Prediction with Dynamic Graph Neural Networks
• Uncertainty quantification in medical image segmentation with normalizing flows
• Multi-layered tensor networks for image classification
• DeepSim: Semantic similarity metrics for learned image registration