1 citations · 1 across the 7 of their papers we have counts for
3 papers · 1 filter
HALO: Hardware-aware quantization with low critical-path-delay weights for LLM acceleration
Rohan Juneja, Shivam Aggarwal, Safeen Huda +2
Quantization is critical for efficiently deploying large language models (LLMs). Yet conventional methods remain hardware-agnostic, limited to bit-width constraints, and do not acc…
TerEffic: Highly Efficient Ternary LLM Inference on FPGA
Chenyang Yin, Zhenyu Bai, Pranav Venkatram +3
Deploying Large Language Models (LLMs) efficiently on edge devices is often constrained by limited memory capacity and high power consumption. Low-bit quantization methods, particu…
DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference
Yujie Zhang, Shivam Aggarwal, Tulika Mitra
Mixture-of-Experts (MoE) models, though highly effective for various machine learning tasks, face significant deployment challenges on memory-constrained devices. While GPUs offer…