90 citations · 229 across the 24 of their papers we have counts for
6 papers · 1 filter
Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format
Chao Fang, Man Shi, Robin Geens +3
The widely-used, weight-only quantized large language models (LLMs), which leverage low-bit integer (INT) weights and retain floating-point (FP) activations, reduce storage require…
Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Cores
Shaobo Ma, Chao Fang, Haikuo Shao +1
Large language models (LLMs) have been widely applied but face challenges in efficient inference. While quantization methods reduce computational demands, ultra-low bit quantizatio…
SPEED: A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference
Chuanning Wang, Chao Fang, Xiao Wu +2
Deploying deep neural networks (DNNs) on those resource-constrained edge platforms is hindered by their substantial computation and storage demands. Quantized multi-precision DNNs,…
Co-Designing Binarized Transformer and Hardware Accelerator for Efficient End-to-End Edge Deployment
Yuhao Ji, Chao Fang, Shaobo Ma +2
Transformer models have revolutionized AI tasks, but their large size hinders real-world deployment on resource-constrained and latency-critical edge devices. While binarized Trans…
A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference
Chuanning Wang, Chao Fang, Xiao Wu +2
RISC-V processors encounter substantial challenges in deploying multi-precision deep neural networks (DNNs) due to their restricted precision support, constrained throughput, and s…
BETA: Binarized Energy-Efficient Transformer Accelerator at the Edge
Yuhao Ji, Chao Fang, Zhongfeng Wang
Existing binary Transformers are promising in edge deployment due to their compact model size, low computational complexity, and considerable inference accuracy. However, deploying…