4 papers · 1 filter
Approximate Speculative Decoding
Yuannuo Feng, Zegang Peng, Yuxin Xie +5
Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel. Under standard greedy verification, decoding stops at the fir…
NANQ: Noise-Floor-Aware Mixed-Precision Non-Uniform Quantization for Analog Compute-in-Memory
Yizhe Chen, Wenshuai Yao, Saiya Wang +6
Analog compute-in-memory (CIM) enables energy-efficient neural network inference, but device variation and read noise can severely degrade low-bit quantized models. Existing CIM-or…
ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
Wenyong Zhou, Yuannuo Feng, Yizhe Chen +6
Large language models (LLMs) with mixture-of-experts (MoE) architectures achieve remarkable scalability by sparsely activating a subset of experts per token, yet their frequent exp…
Extending Straight-Through Estimation for Robust Neural Networks on Analog CIM Hardware
Yuannuo Feng, Wenyong Zhou, Yuexi Lyu +4
Analog Compute-In-Memory (CIM) architectures promise significant energy efficiency gains for neural network inference, but suffer from complex hardware-induced noise that poses maj…