NestedFP: High-Performance, Memory-Efficient Dual-Precision Floating Point Support for LLMs

Haeun Lee, Omin Kwon, Yeonhong Park +1

Meeting service-level objectives (SLOs) in Large Language Models (LLMs) serving is critical, but managing the high variability in load presents a significant challenge. Recent adva…