5 papers · 1 filter
RATIO: Reasoning Analysis and Token-level Inference Optimization for Quantized Reasoning Models
Chengzhu Bao, Xianglong Yan, Tianao Zhang +3
Post-training quantization (PTQ) has become a widely adopted technique for reducing the memory footprint and inference cost of large language models (LLMs). However, recent studies…
WorldDynCache: Risk-Controlled Latent Dynamics Approximation for Diffusion World Model
Leyang Chen, Junyi Wu, Shaoqiu Zhang +1
Diffusion world models generate high-quality futures, but re- peated transformer evaluations make inference prohibitively slow. Existing caches reuse intermediate features, selecti…
EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation
Shu-Hao Zhang, Le-Tong Huang, Xiang-Sheng Deng +5
Quantization has emerged as a mainstream approach for deploying Large Language Models (LLMs) on resource-constrained devices, yet compressing precision below 4-bit typically causes…
Elastic-dLLM: Position Preserving Context Compression and Augmentation of Diffusion LLMs
Junyi Wu, Tianchen Zhao, Shaoqiu Zhang +3
Unlike autoregressive models, which generate one token at a time, dLLMs denoise a chunk of [MASK] tokens jointly and sample one or more tokens per step; despite enabling parallel d…
DQuant: Accurate Low-bit Post-Training Weight Quantization for LLMs
Xianglong Yan, ChengZhu Bao, Zhiteng Li +5
Large language models (LLMs) deliver strong performance, but their high compute and memory costs make deployment difficult in resource-constrained scenarios. Weight-only post-train…