6 papers
QUADS: Stabilizing NVFP4 Reinforcement Learning for MoE via QUantization-error Alignment across Dual Sides
Zhengyang Zhuge, Hao Yu, Xin Wang +4
Rollout generation is a major bottleneck in Reinforcement Learning (RL) for Mixture-of-Experts (MoE) Large Language Models, motivating low-precision rollout acceleration such as FP…
Quantization Meets Spikes: Nearly Lossless Conversion at the First Timestep via Polarity Multi-Spike Mapping
Hangming Zhang, Zheng Li, Chenxiang Ma +4
Spiking neural networks (SNNs) offer advantages in computational efficiency via event-driven computing, compared to traditional artificial neural networks (ANNs). While direct trai…
TriSpec: Ternary Speculative Decoding via Lightweight Proxy Verification
Haoyun Jiang, Junqi He, Feng Hong +8
Inference efficiency in Large Language Models (LLMs) is fundamentally limited by their serial, autoregressive generation, especially as reasoning becomes a key capability and respo…
PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
He Xiao, Runming Yang, Qingyao Yang +6
Post-training quantization (PTQ) of large language models (LLMs) to extremely low bit-widths remains challenging due to the fundamental trade-off between computational efficiency a…
VLCache: Computing 2% Vision Tokens and Reusing 98% for Vision-Language Inference
Shengling Qin, Hao Yu, Chenxin Wu +10
This paper presents VLCache, a cache reuse framework that exploits both Key-Value (KV) cache and encoder cache from prior multimodal inputs to eliminate costly recomputation when t…
FloE: On-the-Fly MoE Inference on Memory-constrained GPU
Yuxin Zhou, Zheng Li, Jun Zhang +5
With the widespread adoption of Mixture-of-Experts (MoE) models, there is a growing demand for efficient inference on memory-constrained devices. While offloading expert parameters…