4 papers
CWind: A Cross-site Router for Large Language Model Inference Serving at Renewable Energy Farms
Tella Rajashekhar Reddy, Atharva Deshmukh, Liangcheng Yu +7
AI power demand is growing at an unprecedented rate while power grids are often ailing and struggle to keep up. Grid expansion comes with high capital expenditure and long-distance…
Improving training time and GPU utilization in geo-distributed language model training
Palak, Tella Rajashekhar Reddy, Bhaskar Kataria +4
The widespread adoption of language models (LMs) has caused a huge surge in demand for GPUs. Training large LMs requires tens of thousands of GPUs and housing them in the same data…
BeLLMan: Controlling LLM Congestion
Tella Rajashekhar Reddy, Atharva Deshmukh, Karan Tandon +3
Large language model (LLM) applications are blindfolded to the infrastructure underneath and generate tokens autoregressively, indifferent to the system load, thus risking inferenc…
AI Greenferencing: Routing AI Inferencing to Green Modular Data Centers with Heron
Tella Rajashekhar Reddy, Palak, Rohan Gandhi +8
AI power demand is growing unprecedentedly thanks to the high power density of AI compute and the emerging inferencing workload. On the supply side, abundant wind power is waiting…