9 papers
ASSERT: Adaptive Stochastic Sampling for Robust Diffusion Models on Analog Compute-in-Memory Hardware
Yuannuo Feng, Yizhe Chen, Wenshuai Yao +4
Diffusion models achieve strong image generation quality but incur high iterative denoising costs. Analog compute-in-memory (CIM) can accelerate matrix-vector multiplications, yet…
Approximate Speculative Decoding
Yuannuo Feng, Zegang Peng, Yuxin Xie +5
Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel. Under standard greedy verification, decoding stops at the fir…
Selective KV Cache Protection for Noise-Resilient LLM Inference on Analog Compute-In-Memory Systems
Yuannuo Feng, Wenyong Zhou, Yuang Ma +5
Analog compute-in-memory (CIM) arrays have emerged as a promising substrate for energy-efficient LLM inference, particularly for weight-stationary computations in linear layers. Ho…
NANQ: Noise-Floor-Aware Mixed-Precision Non-Uniform Quantization for Analog Compute-in-Memory
Yizhe Chen, Wenshuai Yao, Saiya Wang +6
Analog compute-in-memory (CIM) enables energy-efficient neural network inference, but device variation and read noise can severely degrade low-bit quantized models. Existing CIM-or…
Recall Before You Rank: Similarity-Guided Top- Reuse for Efficient Long-Context Attention
Wenshuai Yao, Wenyong Zhou, Hanyong Shao +5
Top- sparse attention reduces the cost of Softmax and value aggregation by attending to only a small subset of key--value (KV) entries. However, identifying this subset still re…
AB-Sparse: Sparse Attention with Adaptive Block Size for Accurate and Efficient Long-Context Inference
Di Liu, Ruitian Wang, Chen Chen +6
As large language models scale to longer contexts, loading the growing KV cache during attention computation becomes a critical bottleneck. Previous work has shown that attention c…
ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
Wenyong Zhou, Yuannuo Feng, Yizhe Chen +6
Large language models (LLMs) with mixture-of-experts (MoE) architectures achieve remarkable scalability by sparsely activating a subset of experts per token, yet their frequent exp…
Extending Straight-Through Estimation for Robust Neural Networks on Analog CIM Hardware
Yuannuo Feng, Wenyong Zhou, Yuexi Lyu +4
Analog Compute-In-Memory (CIM) architectures promise significant energy efficiency gains for neural network inference, but suffer from complex hardware-induced noise that poses maj…
HPD: Hybrid Projection Decomposition for Robust State Space Models on Analog CIM Hardware
Yuannuo Feng, Wenyong Zhou, Yuexi Lyu +4
State Space Models (SSMs) are efficient alternatives to traditional sequence models, excelling at processing long sequences with lower computational complexity. Their reliance on m…