90 citations · 229 across the 24 of their papers we have counts for
25 papers
STELLA: A 16nm Spatio-Temporal Elastic Low-Latency CGRA for Multi-Stage Pipelined Applications
Jun Yin, Chao Fang, Ryan Antonio +4
Emerging non-matrix ML kernels, such as LayerNorm, GeLu, FFT or circular convolutions, demand low-latency, energy-efficient spatial accelerators beyond MatMul-centric arrays. STELL…
ARES: Adaptive Reasoning-Effort Steering for PPA- and Cost-Aware RTL Optimization with LLM Agents
Stef Cuyckens, Mihaela Jivanescu, Jun Yin +2
Large language model (LLM) agents optimize the power, performance, and area (PPA) of register-transfer-level (RTL) designs by iterating over edits, synthesis, and PPA analysis, pay…
HiKV: Hierarchical Importance-Aware KV Cache with Hardware Acceleration for LLM Decoding
Chao Fang, Jun Yin, Man Shi +1
With the rapid adoption of long-context large language models (LLMs), the continuously growing KV cache during decoding has become the critical memory bottleneck. To tackle this ch…
CD-PIM: A High-Bandwidth and Compute-Efficient LPDDR5-Based PIM for Low-Batch LLM Acceleration on Edge-Device
Ye Lin, Chao Fang, Xiaoyong Song +4
Edge deployment of low-batch large language models (LLMs) faces critical memory bandwidth bottlenecks when executing memory-intensive general matrix-vector multiplications (GEMV) o…
A Scheduling Framework for Efficient MoE Inference on Edge GPU-NDP Systems
Qi Wu, Chao Fang, Jiayuan Chen +5
Mixture-of-Experts (MoE) models facilitate edge deployment by decoupling model capacity from active computation, yet their large memory footprint drives the need for GPU systems wi…
Precision-Scalable Microscaling Datapaths with Optimized Reduction Tree for Efficient NPU Integration
Stef Cuyckens, Xiaoling Yi, Robin Geens +4
Emerging continual learning applications necessitate next-generation neural processing unit (NPU) platforms to support both training and inference operations. The promising Microsc…
P3-LLM: An Integrated NPU-PIM Accelerator for Edge LLM Inference Using Hybrid Numerical Formats
Yuzong Chen, Chao Fang, Xilai Dai +4
The substantial memory bandwidth and computational demands of large language models (LLMs) present critical challenges for efficient inference. To tackle this, the literature has e…
iEEG Seizure Detection with a Sparse Hyperdimensional Computing Accelerator
Stef Cuyckens, Ryan Antonio, Chao Fang +1
Implantable devices for reliable intracranial electroencephalography (iEEG) require efficient, accurate, and real-time detection of seizures. Dense hyperdimensional computing (HDC)…
SnipSnap: A Joint Compression Format and Dataflow Co-Optimization Framework for Efficient Sparse LLM Accelerator Design
Junyi Wu, Chao Fang, Zhongfeng Wang
The growing scale of large language models (LLMs) has intensified demands on computation and memory, making efficient inference a key challenge. While sparsity can reduce these cos…
APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Acceleration
Shaobo Ma, Chao Fang, Haikuo Shao +1
Large language models (LLMs) have revolutionized AI applications, yet their enormous computational demands severely limit deployment and real-time performance. Quantization methods…
Enable Lightweight and Precision-Scalable Posit/IEEE-754 Arithmetic in RISC-V Cores for Transprecision Computing
Qiong Li, Chao Fang, Longwei Huang +2
While posit format offers superior dynamic range and accuracy for transprecision computing, its adoption in RISC-V processors is hindered by the lack of a unified solution for ligh…
Efficient Precision-Scalable Hardware for Microscaling (MX) Processing in Robotics Learning
Stef Cuyckens, Xiaoling Yi, Nitish Satya Murthy +2
Autonomous robots require efficient on-device learning to adapt to new environments without cloud dependency. For this edge training, Microscaling (MX) data types offer a promising…
CoDec: Prefix-Shared Decoding Kernel for LLMs
Zhibin Wang, Rui Ning, Chao Fang +12
Prefix-sharing among multiple prompts presents opportunities to combine the operations of the shared prefix, while attention computation in the decode stage, which becomes a critic…
Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format
Chao Fang, Man Shi, Robin Geens +3
The widely-used, weight-only quantized large language models (LLMs), which leverage low-bit integer (INT) weights and retain floating-point (FP) activations, reduce storage require…
Efficient Arbitrary Precision Acceleration for Large Language Models on GPU Tensor Cores
Shaobo Ma, Chao Fang, Haikuo Shao +1
Large language models (LLMs) have been widely applied but face challenges in efficient inference. While quantization methods reduce computational demands, ultra-low bit quantizatio…
SPEED: A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference
Chuanning Wang, Chao Fang, Xiao Wu +2
Deploying deep neural networks (DNNs) on those resource-constrained edge platforms is hindered by their substantial computation and storage demands. Quantized multi-precision DNNs,…
Co-Designing Binarized Transformer and Hardware Accelerator for Efficient End-to-End Edge Deployment
Yuhao Ji, Chao Fang, Shaobo Ma +2
Transformer models have revolutionized AI tasks, but their large size hinders real-world deployment on resource-constrained and latency-critical edge devices. While binarized Trans…
A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference
Chuanning Wang, Chao Fang, Xiao Wu +2
RISC-V processors encounter substantial challenges in deploying multi-precision deep neural networks (DNNs) due to their restricted precision support, constrained throughput, and s…
BETA: Binarized Energy-Efficient Transformer Accelerator at the Edge
Yuhao Ji, Chao Fang, Zhongfeng Wang
Existing binary Transformers are promising in edge deployment due to their compact model size, low computational complexity, and considerable inference accuracy. However, deploying…
Efficient N:M Sparse DNN Training Using Algorithm, Architecture, and Dataflow Co-Design
Chao Fang, Wei Sun, Aojun Zhou +1
Sparse training is one of the promising techniques to reduce the computational cost of DNNs while retaining high accuracy. In particular, N:M fine-grained structured sparsity, wher…
A Precision-Scalable RISC-V DNN Processor with On-Device Learning Capability at the Extreme Edge
Longwei Huang, Chao Fang, Qiong Li +2
Extreme edge platforms, such as in-vehicle smart devices, require efficient deployment of quantized deep neural networks (DNNs) to enable intelligent applications with limited amou…
PDPU: An Open-Source Posit Dot-Product Unit for Deep Learning Applications
Qiong Li, Chao Fang, Zhongfeng Wang
Posit has been a promising alternative to the IEEE-754 floating point format for deep learning applications due to its better trade-off between dynamic range and accuracy. However,…
BEBERT: Efficient and Robust Binary Ensemble BERT
Jiayi Tian, Chao Fang, Haonan Wang +1
Pre-trained BERT models have achieved impressive accuracy on natural language processing (NLP) tasks. However, their excessive amount of parameters hinders them from efficient depl…
An Algorithm-Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformers
Chao Fang, Aojun Zhou, Zhongfeng Wang
The Transformer has been an indispensable staple in deep learning. However, for real-life applications, it is very challenging to deploy efficient Transformers due to immense param…