6 papers
A Framework for Accelerating Transformer Inference on RISC-V for Edge AI
Ajay Kumar M, Vishnu PS, Yike Li +5
This work presents a framework for accelerating transformer-based language models (LMs) on resource-constrained IoT devices. The framework targets compact LMs: BERT-Tiny (B-Ty), Mo…
ZTA-Q: an Open-source RISC-V Platform for Accurate Quantized CNN Inference
Yike Li, Ajay Kumar M, Vishnu PS +4
Low-precision inference is widely adopted in edge AI to reduce computational cost and memory footprint. However, existing open-source accelerator platforms provide limited end-to-e…
ConfigSpec: Profiling-Based Configuration Selection for Distributed Edge--Cloud Speculative LLM Serving
Xiangchen Li, Saeid Ghafouri, Jiakun Fan +3
Speculative decoding enables collaborative Large Language Model (LLM) inference across cloud and edge by separating lightweight token drafting from heavyweight verification. While…
WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batching
Xiangchen Li, Jiakun Fan, Qingyuan Wang +7
As Large Language Models (LLMs) become increasingly accessible to end users, an ever-growing number of inference requests are initiated from edge devices and computed on centralize…
Optimizing video analytics inference pipelines: a case study
Saeid Ghafouri, Yuming Ding, Katerine Diaz Chito +3
Cost-effective and scalable video analytics are essential for precision livestock monitoring, where high-resolution footage and near-real-time monitoring needs from commercial farm…
Polymorph: Energy-Efficient Multi-Label Classification for Video Streams on Embedded Devices
Saeid Ghafouri, Mohsen Fayyaz, Xiangchen Li +4
Real-time multi-label video classification on embedded devices is constrained by limited compute and energy budgets. Yet, video streams exhibit structural properties such as label…