12 papers
Celty: SpMspV GPU Kernel and SIMT Co-Design for Efficient Dual-Sparse LLM Inference
Ruokai Yin, Priyadarshini Panda
Large Language Models (LLMs) increasingly rely on sparsity to reduce inference cost, but most prior work targets a single sparsity source-either weight or activation-and optimizes…
BRIM: Workload-Balanced Dual-Sided Bit-Serial Sparse Inference Accelerator
Varun Manjunath, Ruokai Yin, Donghyun Lee +2
Bit-serial accelerators exploit bit-level sparsity to reduce DNN inference cost, but existing designs exploit sparsity on only one operand, bounding the speedup. Extending sparsity…
KronQ: LLM Quantization via Kronecker-Factored Hessian
Donghyun Lee, Yuhang Li, Ruokai Yin +1
Post-training quantization (PTQ) is a widely adopted technique for compressing large language models (LLMs) without retraining. Most existing second-order PTQ methods, including GP…
Optimal Brain Decomposition for Accurate LLM Low-Rank Approximation
Yuhang Li, Donghyun Lee, Ruokai Yin +1
Low-rank decomposition has emerged as an important problem in Large Language Model (LLM) fine-tuning and inference. Through Singular Value Decomposition (SVD), the weight matrix ca…
TT-SNN: Tensor Train Decomposition for Efficient Spiking Neural Network Training
Donghyun Lee, Ruokai Yin, Youngeun Kim +3
Spiking Neural Networks (SNNs) have gained significant attention as a potentially energy-efficient alternative for standard neural networks with their sparse binary activation. How…
Memba: Membrane-driven Parameter-Efficient Fine-Tuning for Mamba
Donghyun Lee, Yuhang Li, Ruokai Yin +2
State Space Models (SSMs) have emerged as powerful alternatives to attention-based Transformers, with Mamba demonstrating impressive efficiency and scalability. As these models gro…