4 papers
Adaptive Parallel Monte Carlo Tree Search for Efficient Test-time Compute Scaling
Hongbeen Kim, Juhyun Lee, Sanghyeon Lee +2
Monte Carlo Tree Search (MCTS) is an effective test-time compute scaling (TTCS) method for improving the reasoning performance of large language models, but its highly variable exe…
Adaptive Migration Decision for Multi-Tenant Memory Systems
Hyungjun Cho, Igjae Kim, Kwanghoon Choi +5
Tiered memory systems consisting of fast small memory and slow large memory have emerged to provide high capacity memory in a cost-effective way. The effectiveness of tiered memory…
Hardware-based Heterogeneous Memory Management for Large Language Model Inference
Soojin Hwang, Jungwoo Kim, Sanghyeon Lee +2
A large language model (LLM) is one of the most important emerging machine learning applications nowadays. However, due to its huge model size and runtime increase of the memory fo…
Efficient LLM Inference with Activation Checkpointing and Hybrid Caching
Sanghyeon Lee, Hongbeen Kim, Soojin Hwang +3
Recent large language models (LLMs) with enormous model sizes use many GPUs to meet memory capacity requirements incurring substantial costs for token generation. To provide cost-e…