Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillation

Fangbo Tu, Junhua Zhao, Chi Liu +4

Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production enviro…