6 papers
Approximate Speculative Decoding
Yuannuo Feng, Zegang Peng, Yuxin Xie +5
Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel. Under standard greedy verification, decoding stops at the fir…
NANQ: Noise-Floor-Aware Mixed-Precision Non-Uniform Quantization for Analog Compute-in-Memory
Yizhe Chen, Wenshuai Yao, Saiya Wang +6
Analog compute-in-memory (CIM) enables energy-efficient neural network inference, but device variation and read noise can severely degrade low-bit quantized models. Existing CIM-or…
Selective KV Cache Protection for Noise-Resilient LLM Inference on Analog Compute-In-Memory Systems
Yuannuo Feng, Wenyong Zhou, Yuang Ma +5
Analog compute-in-memory (CIM) arrays have emerged as a promising substrate for energy-efficient LLM inference, particularly for weight-stationary computations in linear layers. Ho…
ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
Wenyong Zhou, Yuannuo Feng, Yizhe Chen +6
Large language models (LLMs) with mixture-of-experts (MoE) architectures achieve remarkable scalability by sparsely activating a subset of experts per token, yet their frequent exp…
Extending Straight-Through Estimation for Robust Neural Networks on Analog CIM Hardware
Yuannuo Feng, Wenyong Zhou, Yuexi Lyu +4
Analog Compute-In-Memory (CIM) architectures promise significant energy efficiency gains for neural network inference, but suffer from complex hardware-induced noise that poses maj…
HPD: Hybrid Projection Decomposition for Robust State Space Models on Analog CIM Hardware
Yuannuo Feng, Wenyong Zhou, Yuexi Lyu +4
State Space Models (SSMs) are efficient alternatives to traditional sequence models, excelling at processing long sequences with lower computational complexity. Their reliance on m…