6 papers
From Characterization to Microarchitecture: Designing an Elegant and Reliable BFP-Based NPU
Jie Zhang, Jiapeng Guan, Hao Zhou +4
Block Floating-Point (BFP) is emerging as an attractive data format for edge Neural Processing Units (NPUs), combining wide dynamic range with high hardware efficiency. However, it…
NLI:Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inference
Jiangyong Yu, Xiaomeng Han, Xing Hu +3
Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of tasks, but their deployment is often constrained by substantial memory footprints and c…
Re-thinking Memory-Bound Limitations in CGRAs
Xiangfeng Liu, Zhe Jiang, Anzhen Zhu +4
Coarse-Grained Reconfigurable Arrays (CGRAs) are specialized accelerators commonly employed to boost performance in workloads with iterative structures. Existing research typically…
BBAL: A Bidirectional Block Floating Point-Based Quantisation Accelerator for Large Language Models
Xiaomeng Han, Yuan Cheng, Jing Wang +6
Large language models (LLMs), with their billions of parameters, pose substantial challenges for deployment on edge devices, straining both memory capacity and computational resour…
NVR: Vector Runahead on NPUs for Sparse Memory Access
Hui Wang, Zhengpeng Zhao, Jing Wang +11
Deep Neural Networks are increasingly leveraging sparsity to reduce the scaling up of model parameter size. However, reducing wall-clock time through sparsity and pruning remains c…
Pushing the Limits of BFP on Narrow Precision LLM Inference
Hui Wang, Yuan Cheng, Xiaomeng Han +3
The substantial computational and memory demands of Large Language Models (LLMs) hinder their deployment. Block Floating Point (BFP) has proven effective in accelerating linear ope…