4 papers
MOLT: A Fine-Grained GPU Memory Sharing System for LLM Serving with Opportunistic Fine-Tuning
Jaehoon Yang, Yongbeom Kim, Hojoon Kim +2
Large language model (LLM) serving scales its replica count with the request load, yet GPU memory still stands idle inside the replicas. Adding a replica takes minutes, while the m…
AgenticCache: Cache-Driven Asynchronous Planning for Embodied AI Agents
Hojoon Kim, Yuheng Wu, Thierry Tambe
Embodied AI agents increasingly rely on large language models (LLMs) for planning, yet per-step LLM calls impose severe latency and cost. In this paper, we show that embodied tasks…
Déjà Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reuse
Jinwoo Hwang, Daeun Kim, Sangyeop Lee +8
Recently, Video-Language Models (VideoLMs) have demonstrated remarkable capabilities, offering significant potential for flexible and powerful video query systems. These models typ…
DecDEC: A Systems Approach to Advancing Low-Bit LLM Quantization
Yeonhong Park, Jake Hyun, Hojoon Kim +1
Quantization of Large Language Models (LLMs) has recently gained popularity, particularly for on-device settings with limited hardware resources. While efficient, quantization inev…