53 citations · 71 across the 16 of their papers we have counts for
3 papers · 1 filter
Dynamic Rebatching for Efficient Early-Exit Inference with DREX
Xuting Liu, Daniel Alexander, Siva Kesava Reddy Kakarla +2
Early-Exit (EE) is a Large Language Model (LLM) architecture that accelerates inference by allowing easier tokens to be generated using only a subset of the model's layers. However…
A Performance Analyzer for a Public Cloud's ML-Augmented VM Allocator
Roozbeh Bostandoost, Pooria Namyar, Siva Kesava Reddy Kakarla +8
Cloud operators increasingly deploy multiple ML models in their VM allocation pipelines. In such settings, individually benign predictions can shift and compound, severely degradin…
Robust Heuristic Algorithm Design with LLMs
Pantea Karimi, Dany Rouhana, Pooria Namyar +3
We posit that we can generate more robust and performant heuristics if we augment approaches using LLMs for heuristic design with tools that explain why heuristics underperform and…