8 papers
Learning Perturbation Robust Policies for LLM Agents with Stable Optimization
Pengxin Wang, Yuanzhe LI, Yuxin Ren +2
Reinforcement learning (RL) has become an effective post-training paradigm for long-horizon large language model (LLM) agents. However, we find that the resulting policies can be s…
From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning
Pan Wang, Siwei Song, Hui Ji +12
The rapid expansion of multimodal models has surfaced formidable bottlenecks in computation, memory, and deployment, catalyzing the rise of Efficient Multimodal Learning (EML) as a…
TASQ: Temporal-Adaptive Bit Sparsification Quantization for Diffusion Models
Seokho Han, Dongwei Wang, Jinhee Kim +4
Static quantization assigns one weight precision to every denoising step. To preserve quality, that precision must accommodate the most quantization-sensitive step, even though man…
MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM
Dongwei Wang, Jinhee Kim, Seokho Han +8
Dynamic runtime latency and memory constraints necessitate flexible large language model (LLM) deployment, where an LLM can be inferred with various quantization precisions based o…
FIER: Fine-Grained and Efficient KV Cache Retrieval for Long-context LLM Inference
Dongwei Wang, Zijie Liu, Song Wang +5
The Key-Value (KV) cache reading latency increases significantly with context lengths, hindering the efficiency of long-context LLM inference. To address this, previous works propo…
MSQ: Memory-Efficient Bit Sparsification Quantization
Seokho Han, Seoyeon Yoon, Jinhee Kim +4
As deep neural networks (DNNs) see increased deployment on mobile and edge devices, optimizing model efficiency has become crucial. Mixed-precision quantization is widely favored,…
FAR: Function-preserving Attention Replacement for IMC-friendly Inference
Yuxin Ren, Maxwell D Collins, Miao Hu +1
While transformers dominate modern vision and language models, their attention mechanism remains poorly suited for in-memory computing (IMC) devices due to intensive activation-to-…
Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization
Dongwei Wang, Huanrui Yang
Quantization is a critical step to enable efficient LLM serving under limited resource. However, previous research observes that certain weights in the LLM, known as outliers, are…