Power Hungry Processing: Watts Driving the Cost of AI Deployment?

Abstract

Recent years have seen a surge in the popularity of commercial AI products based on generative, multi-purpose AI systems promising a unified approach to building machine learning (ML) models into technology. However, this ambition of ``generality'' comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. In this work, we propose the first systematic comparison of the ongoing inference cost of various categories of ML systems, covering both task-specific (i.e. finetuned models that carry out a single task) and `general-purpose' models, (i.e. those trained for multiple tasks). We measure deployment cost as the amount of energy and carbon required to perform 1,000 inferences on representative benchmark dataset using these models. We find that multi-purpose, generative architectures are orders of magnitude more expensive than task-specific systems for a variety of tasks, even when controlling for the number of model parameters. We conclude with a discussion around the current trend of deploying multi-purpose generative ML systems, and caution that their utility should be more intentionally weighed against increased costs in terms of energy and emissions. All the data from our study can be accessed via an interactive demo to carry out further exploration and analysis.

References in corpus (2)

• Scaling Instruction-Finetuned Language Models
• The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink

Cited by in corpus (16)

• From Efficiency Gains to Rebound Effects: The Problem of Jevons' Paradox in AI's Polarized Environmental Debate
• Situating the social issues of image generation models in the model life cycle: a sociotechnical approach
• LACS: Learning-Augmented Algorithms for Carbon-Aware Resource Scaling with Uncertain Demand
• Greening AI-enabled Systems with Software Engineering: A Research Agenda for Environmentally Sustainable AI Practices
• GenAI as Digital Plastic: Understanding Synthetic Media Through Critical AI Literacy
• GPTFootprint: Increasing Consumer Awareness of the Environmental Impacts of LLMs
• ALPINE: An adaptive language-agnostic pruning method for language models for code
• How Do Companies Manage the Environmental Sustainability of AI? An Interview Study About Green AI Efforts and Regulations
• Generative AI for Research Data Processing: Lessons Learnt From Three Use Cases
• Security practices in AI development
• Ground-Truthing AI Energy Consumption: Validating CodeCarbon Against External Measurements
• Notes On Writing Effective Empirical Software Engineering Papers: An Opinionated Primer
• Mitigating the Carbon Footprint of Chatbots as Consumers
• Evaluating Sakana's AI Scientist: Bold Claims, Mixed Results, and a Promising Future?
• Vectorizing string entries for data processing on tables: when are larger language models better?
• Analysing Environmental Efficiency in AI for X-Ray Diagnosis