4 papers
HALO: Hardware-aware quantization with low critical-path-delay weights for LLM acceleration
Rohan Juneja, Shivam Aggarwal, Safeen Huda +2
Quantization is critical for efficiently deploying large language models (LLMs). Yet conventional methods remain hardware-agnostic, limited to bit-width constraints, and do not acc…
Condensed Data Expansion Using Model Inversion for Knowledge Distillation
Kuluhan Binici, Shivam Aggarwal, Cihan Acar +4
Condensed datasets offer a compact representation of larger datasets, but training models directly on them or using them to enhance model performance through knowledge distillation…
DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference
Yujie Zhang, Shivam Aggarwal, Tulika Mitra
Mixture-of-Experts (MoE) models, though highly effective for various machine learning tasks, face significant deployment challenges on memory-constrained devices. While GPUs offer…
TerEffic: Highly Efficient Ternary LLM Inference on FPGA
Chenyang Yin, Zhenyu Bai, Pranav Venkatram +3
Deploying Large Language Models (LLMs) efficiently on edge devices is often constrained by limited memory capacity and high power consumption. Low-bit quantization methods, particu…