180 citations · 520 across the 35 of their papers we have counts for
7 papers · 1 filter
SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Serving
Gunho Park, Kyoungho Jeun, Juntaek Oh +3
Mixture-of-experts (MoE) models activate few experts per token, yet batched decoding can access nearly the entire expert pool, making expert-weight traffic a major bottleneck. Expe…
PASCAL: A Phase-Aware Scheduling Algorithm for Serving Reasoning-based Large Language Models
Eunyeong Cho, Jehyeon Bang, Ranggi Hwang +1
The emergence of reasoning-based LLMs leveraging Chain-of-Thought (CoT) inference introduces new serving challenges, as their extended reasoning phases delay user-visible output an…
The Cost of Dynamic Reasoning: Demystifying AI Agents and Test-Time Scaling from an AI Infrastructure Perspective
Jiin Kim, Byeongjun Shin, Jinha Chung +1
Large-language-model (LLM)-based AI agents have recently showcased impressive versatility by employing dynamic reasoning, an adaptive, multi-step process that coordinates with exte…
vTrain: A Simulation Framework for Evaluating Cost-effective and Compute-optimal Large Language Model Training
Jehyeon Bang, Yujeong Choi, Myeongwoo Kim +2
As large language models (LLMs) become widespread in various application domains, a critical challenge the AI community is facing is how to train these large AI models in a cost-ef…
TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learning
Youngeun Kwon, Yunjae Lee, Minsoo Rhu
Recent studies from several hyperscalars pinpoint to embedding layers as the most memory-intensive deep learning (DL) algorithm being deployed in today's datacenters. This paper ad…
Structurally Sparsified Backward Propagation for Faster Long Short-Term Memory Training
Maohua Zhu, Jason Clemons, Jeff Pool +3
Exploiting sparsity enables hardware systems to run neural networks faster and more energy-efficiently. However, most prior sparsity-centric optimization techniques only accelerate…
Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks
Minsoo Rhu, Mike O'Connor, Niladrish Chatterjee +2
Popular deep learning frameworks require users to fine-tune their memory usage so that the training data of a deep neural network (DNN) fits within the GPU physical memory. Prior w…