1 citations · 1 across the 4 of their papers we have counts for
5 papers
Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design
Seonghun Jung, Sieun Moon, Jiyoung Jeong +2
Vision-language-action (VLA) models translate multimodal observations into low-level robot actions. During robot operation, each control period sets an inference deadline, and over…
Adaptive Parallel Monte Carlo Tree Search for Efficient Test-time Compute Scaling
Hongbeen Kim, Juhyun Lee, Sanghyeon Lee +2
Monte Carlo Tree Search (MCTS) is an effective test-time compute scaling (TTCS) method for improving the reasoning performance of large language models, but its highly variable exe…
Adaptive Migration Decision for Multi-Tenant Memory Systems
Hyungjun Cho, Igjae Kim, Kwanghoon Choi +5
Tiered memory systems consisting of fast small memory and slow large memory have emerged to provide high capacity memory in a cost-effective way. The effectiveness of tiered memory…
Hardware-based Heterogeneous Memory Management for Large Language Model Inference
Soojin Hwang, Jungwoo Kim, Sanghyeon Lee +2
A large language model (LLM) is one of the most important emerging machine learning applications nowadays. However, due to its huge model size and runtime increase of the memory fo…
Throughput-Oriented LLM Inference via KV-Activation Hybrid Caching with a Single GPU
Sanghyeon Lee, Hongbeen Kim, Soojin Hwang +3
Large language models (LLMs) with growing model sizes use many GPUs to meet memory capacity requirements, incurring substantial costs for token generation. To provide cost-effectiv…