5 papers
Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference
Sunghyun Wee, Geunjae Choi, Hyeonjin Kim +2
Large Language Models (LLMs) demand efficient inference, where the multiply-accumulate (MAC) operations in linear projections dominate compute and energy cost. Post-training quanti…
Zero-Shot Quantization for Object Detectors using Off-the-Shelf Generative Models
Hyunho Lee, Kyomin Hwang, Hyeonjin Kim +3
With an increasing number of Object Detection (OD) models being deployed on edge devices, Zero-Shot Quantization for OD (ZSQ-OD) aims to quantize these models when access to the or…
ReSpinQuant: Efficient Layer-Wise LLM Quantization via Subspace Residual Rotation Approximation
Suyoung Kim, Sunghyun Wee, Hyeonjin Kim +3
Rotation-based Post-Training Quantization (PTQ) has emerged as a promising solution for mitigating activation outliers in the quantization of Large Language Models (LLMs). Global r…
Safety-Preserving PTQ via Contrastive Alignment Loss
Sunghyun Wee, Suyoung Kim, Hyeonjin Kim +2
Post-Training Quantization (PTQ) has become the de-facto standard for efficient LLM deployment, yet its optimization objective remains fundamentally incomplete. Standard PTQ method…
Uncovering the Potential Risks in Unlearning: Danger of English-only Unlearning in Multilingual LLMs
Kyomin Hwang, Hyeonjin Kim, Seungyeon Kim +2
There have been a couple of studies showing that attempting to erase multilingual knowledge using only English data is insufficient for multilingual LLMs. However, their analyses r…