5 papers · 1 filter
KronQ: LLM Quantization via Kronecker-Factored Hessian
Donghyun Lee, Yuhang Li, Ruokai Yin +1
Post-training quantization (PTQ) is a widely adopted technique for compressing large language models (LLMs) without retraining. Most existing second-order PTQ methods, including GP…
BRIM: Workload-Balanced Dual-Sided Bit-Serial Sparse Inference Accelerator
Varun Manjunath, Ruokai Yin, Donghyun Lee +2
Bit-serial accelerators exploit bit-level sparsity to reduce DNN inference cost, but existing designs exploit sparsity on only one operand, bounding the speedup. Extending sparsity…
OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers
Donghyun Lee, Jitesh Chavan, Duy Nguyen +5
Diffusion transformers (DiTs) achieve state-of-the-art image and video generation, but their multi-step sampling and growing parameter count make inference expensive. Post-training…
Optimal Brain Decomposition for Accurate LLM Low-Rank Approximation
Yuhang Li, Donghyun Lee, Ruokai Yin +1
Low-rank decomposition has emerged as an important problem in Large Language Model (LLM) fine-tuning and inference. Through Singular Value Decomposition (SVD), the weight matrix ca…
MD-SNN: Membrane Potential-aware Distillation on Quantized Spiking Neural Network
Donghyun Lee, Abhishek Moitra, Youngeun Kim +2
Spiking Neural Networks (SNNs) offer a promising and energy-efficient alternative to conventional neural networks, thanks to their sparse binary activation. However, they face chal…