Hybrid Systolic Array Accelerator with Optimized Dataflow for Edge Large Language Model Inference

Chun-Ting Chen, HanGyeol Mun, Jian Meng +2

Edge inference for large language models (LLM) offers secure, low-latency, and cost-effective inference solutions. We emphasize that an edge accelerator should achieve high area ef…