4 papers · 1 filter
QUADS: Stabilizing NVFP4 Reinforcement Learning for MoE via QUantization-error Alignment across Dual Sides
Zhengyang Zhuge, Hao Yu, Xin Wang +4
Rollout generation is a major bottleneck in Reinforcement Learning (RL) for Mixture-of-Experts (MoE) Large Language Models, motivating low-precision rollout acceleration such as FP…
TriSpec: Ternary Speculative Decoding via Lightweight Proxy Verification
Haoyun Jiang, Junqi He, Feng Hong +8
Inference efficiency in Large Language Models (LLMs) is fundamentally limited by their serial, autoregressive generation, especially as reasoning becomes a key capability and respo…
PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
He Xiao, Runming Yang, Qingyao Yang +6
Post-training quantization (PTQ) of large language models (LLMs) to extremely low bit-widths remains challenging due to the fundamental trade-off between computational efficiency a…
FloE: On-the-Fly MoE Inference on Memory-constrained GPU
Yuxin Zhou, Zheng Li, Jun Zhang +5
With the widespread adoption of Mixture-of-Experts (MoE) models, there is a growing demand for efficient inference on memory-constrained devices. While offloading expert parameters…