9 papers
LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression
Zishan Shao, Lixun Zhang, Kangning Cui +10
SVD-based low-rank compression has become a fast-growing direction for reducing the memory and computational cost of large language models (LLMs). However, meaningful comparison ac…
TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models
Seokho Han, Dongwei Wang, Jinhee Kim +4
Static quantization assigns one weight precision to every denoising step. To preserve quality, that precision must accommodate the most quantization-sensitive step, even though man…
FlashSVD v1.5: Making Low-Rank Transformers Inference Actually Fast
Wenhao Wu, Zishan Shao, Kangning Cui +5
SVD-based Low-rank compression reduces transformer parameters and nominal FLOPs, but these savings often translate poorly into real LLM serving speedups. We show that this gap is l…
MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM
Dongwei Wang, Jinhee Kim, Seokho Han +8
Dynamic runtime latency and memory constraints necessitate flexible large language model (LLM) deployment, where an LLM can be inferred with various quantization precisions based o…
Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling
Jinhee Kim, Jae Jun An, Kang Eun Jeon +1
Multi-bit quantization networks enable flexible deployment of deep neural networks by supporting multiple precision levels within a single model. However, existing approaches suffe…
Row-Column Hybrid Grouping for Fault-Resilient Multi-Bit Weight Representation on IMC Arrays
Kang Eun Jeon, Sangheum Yeon, Jinhee Kim +3
This paper addresses two critical challenges in analog In-Memory Computing (IMC) systems that limit their scalability and deployability: the computational unreliability caused by s…
MSQ: Memory-Efficient Bit Sparsification Quantization
Seokho Han, Seoyeon Yoon, Jinhee Kim +4
As deep neural networks (DNNs) see increased deployment on mobile and edge devices, optimizing model efficiency has become crucial. Mixed-precision quantization is widely favored,…
TruncQuant: Truncation-Ready Quantization for DNNs with Flexible Weight Bit Precision
Jinhee Kim, Seoyeon Yoon, Taeho Lee +3
The deployment of deep neural networks on edge devices is a challenging task due to the increasing complexity of state-of-the-art models, requiring efforts to reduce model size and…
MEMHD: Memory-Efficient Multi-Centroid Hyperdimensional Computing for Fully-Utilized In-Memory Computing Architectures
Do Yeong Kang, Yeong Hwan Oh, Chanwook Hwang +3
The implementation of Hyperdimensional Computing (HDC) on In-Memory Computing (IMC) architectures faces significant challenges due to the mismatch between highdimensional vectors a…