1 citations · 1 across the 3 of their papers we have counts for
3 papers · 1 filter
Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation
Joon Ha Kim, Geon-Woo Kim, Anoop Rachakonda +1
Selecting the optimal LLM inference configuration requires evaluation across hardware, serving engines, attention backends, and model architectures, since no single choice performs…
Toward Cost-Efficient Serving of Mixture-of-Experts with Asynchrony
Shaoyu Wang, Guangrong He, Geon-Woo Kim +2
Mixture-of-Experts (MoE) architectures offer the promise of larger model capacity without the prohibitive costs of fully dense designs. However, in real-world inference serving, lo…
Lovelock: Towards Smart NIC-hosted Clusters
Seo Jin Park, Ramesh Govindan, Kai Shen +4
Traditional cluster designs were originally server-centric, and have evolved recently to support hardware acceleration and storage disaggregation. In applications that leverage acc…