7 papers
QuantClaw: Precision Where It Matters for OpenClaw
Manyi Zhang, Ji-Fu Li, Zhongao Sun +5
Autonomous agent systems such as OpenClaw introduce significant efficiency challenges due to long-context inputs and multi-turn reasoning. This results in prohibitively high comput…
HAWK: Head Importance-Aware Visual Token Pruning in Multimodal Models
Qihui Zhu, Tao Zhang, Yuchen Wang +9
In multimodal large language models (MLLMs), the surge of visual tokens significantly increases the inference time and computational overhead, making them impractical for real-time…
BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization
Ji-Fu Li, Manyi Zhang, Xiaobo Xia +4
Microscaling floating-point (MXFP) formats have emerged as a promising standard for deploying Multi-modal Large Language Models (MLLMs) and Large Language Models (LLMs) on modern a…
FreeAct: Freeing Activations for LLM Quantization
Xiaohao Liu, Xiaobo Xia, Manyi Zhang +6
Quantization is pivotal for mitigating the significant memory and computational overhead of Large Language Models (LLMs). While emerging transformation-based methods have successfu…
What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study
Keyu Lv, Manyi Zhang, Xiaobo Xia +6
Reasoning models excel at complex tasks such as coding and mathematics, yet their inference is often slow and token-inefficient. To improve the inference efficiency, post-training…
Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats
Manyi Zhang, Ji-Fu Li, Zhongao Sun +4
Microscaling Floating-Point (MXFP) has emerged as a promising low-precision format for large language models (LLMs). Despite various post-training quantization (PTQ) algorithms bei…