72 citations · 351 across the 14 of their papers we have counts for
14 papers
EcoLife: Carbon-Aware Serverless Function Scheduling for Sustainable Computing
Yankai Jiang, Rohan Basu Roy, Baolin Li +1
This work introduces ECOLIFE, the first carbon-aware serverless function scheduler to co-optimize carbon footprint and performance. ECOLIFE builds on the key insight of intelligent…
LLM Inference Serving: Survey of Recent Advances and Opportunities
Baolin Li, Yankai Jiang, Vijay Gadepally +1
This survey offers a comprehensive overview of recent advancements in Large Language Model (LLM) serving systems, focusing on research since the year 2023. We specifically examine…
Toward Sustainable GenAI using Generation Directives for Carbon-Friendly Large Language Model Inference
Baolin Li, Yankai Jiang, Vijay Gadepally +1
The rapid advancement of Generative Artificial Intelligence (GenAI) across diverse sectors raises significant environmental concerns, notably the carbon emissions from their cloud…
Sustainable Supercomputing for AI: GPU Power Capping at HPC Scale
Dan Zhao, Siddharth Samsi, Joseph McDonald +5
As research and deployment of AI grows, the computational burden to support and sustain its progress inevitably does too. To train or fine-tune state-of-the-art models in NLP, comp…
From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference
Siddharth Samsi, Dan Zhao, Joseph McDonald +7
Large language models (LLMs) have exploded in popularity due to their new generative capabilities that go far beyond prior state-of-the-art. These technologies are increasingly bei…
Toward Sustainable HPC: Carbon Footprint Estimation and Environmental Implications of HPC Systems
Baolin Li, Rohan Basu Roy, Daniel Wang +3
The rapid growth in demand for HPC systems has led to a rise in carbon footprint, which requires urgent intervention. In this work, we present a comprehensive analysis of the carbo…
Clover: Toward Sustainable AI with Carbon-Aware Machine Learning Inference Service
Baolin Li, Siddharth Samsi, Vijay Gadepally +1
This paper presents a solution to the challenge of mitigating carbon emissions from hosting large-scale machine learning (ML) inference services. ML inference is critical to modern…
KAIROS: Building Cost-Efficient Machine Learning Inference Systems with Heterogeneous Cloud Resources
Baolin Li, Siddharth Samsi, Vijay Gadepally +1
Online inference is becoming a key service product for many businesses, deployed in cloud platforms to meet customer demands. Despite their revenue-generation capability, these ser…
RIBBON: Cost-Effective and QoS-Aware Deep Learning Model Inference using a Diverse Pool of Cloud Computing Instances
Baolin Li, Rohan Basu Roy, Tirthak Patel +3
Deep learning model inference is a key service in many businesses and scientific discovery processes. This paper introduces RIBBON, a novel deep learning inference serving system t…
MISO: Exploiting Multi-Instance GPU Capability on Multi-Tenant Systems for Machine Learning
Baolin Li, Tirthak Patel, Siddarth Samsi +2
GPU technology has been improving at an expedited pace in terms of size and performance, empowering HPC and AI/ML researchers to advance the scientific discovery process. However,…
Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models
Joseph McDonald, Baolin Li, Nathan Frey +3
The energy requirements of current natural language processing models continue to grow at a rapid, unsustainable pace. Recent works highlighting this problem conclude there is an u…
The MIT Supercloud Workload Classification Challenge
Benny J. Tang, Qiqi Chen, Matthew L. Weiss +24
High-Performance Computing (HPC) centers and cloud providers support an increasingly diverse set of applications on heterogenous hardware. As Artificial Intelligence (AI) and Machi…
Benchmarking Resource Usage for Efficient Distributed Deep Learning
Nathan C. Frey, Baolin Li, Joseph McDonald +6
Deep learning (DL) workflows demand an ever-increasing budget of compute and energy in order to achieve outsized gains. Neural architecture searches, hyperparameter sweeps, and rap…
The MIT Supercloud Dataset
Siddharth Samsi, Matthew L Weiss, David Bestor +24
Artificial intelligence (AI) and Machine learning (ML) workloads are an increasingly larger share of the compute workloads in traditional High-Performance Computing (HPC) centers a…