13 papers
SoloQ: Calibration-Free Quantization for Diffusion Language Models
Donghyun Lee, Arkapravo Ghosh, Varun Manjunath +5
Diffusion large language models dLLMs) have emerged as a promising alternative to autoregressive language models through bidirectional diffusion-based token generation. However, th…
Celty: SpMSpV GPU Kernel and SIMT Co-Design for Efficient Dual-Sparse LLM Inference
Ruokai Yin, Priyadarshini Panda
Large Language Models (LLMs) increasingly rely on sparsity to cut inference cost, but most prior work exploits a single sparsity source and targets batched multi-user inference. Du…
KronQ: LLM Quantization via Kronecker-Factored Hessian
Donghyun Lee, Yuhang Li, Ruokai Yin +1
Post-training quantization (PTQ) is a widely adopted technique for compressing large language models (LLMs) without retraining. Most existing second-order PTQ methods, including GP…
BRIM: Workload-Balanced Dual-Sided Bit-Serial Sparse Inference Accelerator
Varun Manjunath, Ruokai Yin, Donghyun Lee +2
Bit-serial accelerators exploit bit-level sparsity to reduce DNN inference cost, but existing designs exploit sparsity on only one operand, bounding the speedup. Extending sparsity…
OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers
Donghyun Lee, Jitesh Chavan, Duy Nguyen +5
Diffusion transformers (DiTs) achieve state-of-the-art image and video generation, but their multi-step sampling and growing parameter count make inference expensive. Post-training…
Optimal Brain Decomposition for Accurate LLM Low-Rank Approximation
Yuhang Li, Donghyun Lee, Ruokai Yin +1
Low-rank decomposition has emerged as an important problem in Large Language Model (LLM) fine-tuning and inference. Through Singular Value Decomposition (SVD), the weight matrix ca…
QuRL: Efficient Reinforcement Learning with Quantized Rollout
Yuhang Li, Reena Elangovan, Xin Dong +2
Reinforcement learning with verifiable rewards (RLVR) has become a trending paradigm for training reasoning large language models (LLMs). However, due to the autoregressive decodin…
MD-SNN: Membrane Potential-aware Distillation on Quantized Spiking Neural Network
Donghyun Lee, Abhishek Moitra, Youngeun Kim +2
Spiking Neural Networks (SNNs) offer a promising and energy-efficient alternative to conventional neural networks, thanks to their sparse binary activation. However, they face chal…
SpikePool: Event-driven Spiking Transformer with Pooling Attention
Donghyun Lee, Alex Sima, Yuhang Li +2
Building on the success of transformers, Spiking Neural Networks (SNNs) have increasingly been integrated with transformer architectures, leading to spiking transformers that demon…
OpenWorldSAM: Extending SAM2 for Universal Image Segmentation with Language Prompts
Shiting Xiao, Rishabh Kabra, Yuhang Li +3
The ability to segment objects based on open-ended language prompts remains a critical challenge, requiring models to ground textual semantics into precise spatial masks while hand…
DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs
Ruokai Yin, Yuhang Li, Donghyun Lee +1
Large language models (LLMs) deliver strong performance but are difficult to deploy due to high memory and compute costs. While pruning reduces these demands, most methods ignore a…
Memba: Membrane-driven Parameter-Efficient Fine-Tuning for Mamba
Donghyun Lee, Yuhang Li, Ruokai Yin +2
State Space Models (SSMs) have emerged as powerful alternatives to attention-based Transformers, with Mamba demonstrating impressive efficiency and scalability. As these models gro…
GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration
Yuhang Li, Ruokai Yin, Donghyun Lee +2
We introduce GPTAQ, a novel finetuning-free quantization method for compressing large-scale transformer architectures. Unlike the previous GPTQ method, which independently calibrat…