TerEffic: Highly Efficient Ternary LLM Inference on FPGA

Chenyang Yin, Zhenyu Bai, Pranav Venkatram +3

Deploying Large Language Models (LLMs) efficiently on edge devices is often constrained by limited memory capacity and high power consumption. Low-bit quantization methods, particu…