180 citations · 520 across the 35 of their papers we have counts for
6 papers · 1 filter
SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Serving
Gunho Park, Kyoungho Jeun, Juntaek Oh +3
Mixture-of-experts (MoE) models activate few experts per token, yet batched decoding can access nearly the entire expert pool, making expert-weight traffic a major bottleneck. Expe…
Agent-X: Full Pipeline Acceleration of On-device AI Agents
Jinha Chung, Byeongjun Shin, Jiin Kim +1
LLM-based agents deliver state-of-the-art performance across tasks but incur high end-to-end latency on edge devices. We introduce Agent-X, a software-only, accuracy-preserving fra…
SpecMoE: A Fast and Efficient Mixture-of-Experts Inference via Self-Assisted Speculative Decoding
Jehyeon Bang, Eunyeong Cho, Ranggi Hwang +2
The Mixture-of-Experts (MoE) architecture has emerged as a promising approach to mitigate the rising computational costs of large language models (LLMs) by selectively activating p…
SwarmIO: Towards 100 Million IOPS SSD Emulation for Next-generation GPU-centric Storage Systems
Hyeseong Kim, Gwangoo Yeo, Minsoo Rhu
GPU-initiated I/O has emerged as a key mechanism for achieving high-throughput storage access by leveraging massive GPU thread-level parallelism, while recent industry trends point…
PASCAL: A Phase-Aware Scheduling Algorithm for Serving Reasoning-based Large Language Models
Eunyeong Cho, Jehyeon Bang, Ranggi Hwang +1
The emergence of reasoning-based LLMs leveraging Chain-of-Thought (CoT) inference introduces new serving challenges, as their extended reasoning phases delay user-visible output an…
ZipFlow: a Compiler-based Framework to Unleash Compressed Data Movement for Modern GPUs
Gwangoo Yeo, Zhiyang Shen, Wei Cui +5
In GPU-accelerated data analytics, the overhead of data transfer from CPU to GPU becomes a performance bottleneck when the data scales beyond GPU memory capacity due to the limited…