Showing 2025Show all
2 papers · 1 filter
cs.LG2025
DP-LLM: Runtime Model Adaptation with Dynamic Layer-wise Precision Assignment
Sangwoo Kwon, Seong Hoon Seo, Jae W. Lee +1
How can we effectively handle queries for on-device large language models (LLMs) with varying runtime constraints, such as latency and accuracy? Multi-scale quantization addresses…
cs.DC2025
NestedFP: High-Performance, Memory-Efficient Dual-Precision Floating Point Support for LLMs
Haeun Lee, Omin Kwon, Yeonhong Park +1
Meeting service-level objectives (SLOs) in Large Language Models (LLMs) serving is critical, but managing the high variability in load presents a significant challenge. Recent adva…