11 papers
BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration
Chence Yang, Ningxi Cheng, Arash Akbari +9
Speculative decoding accelerates autoregressive generation by using a lightweight draft to propose multiple tokens for parallel verification. However, existing methods often requir…
Ask or Answer: A Decision Framework for Multi-Turn Health Misinformation Intervention
Xiaoying Song, Anirban Saha Anik, Jinyu Liu +3
Correcting health misinformation in dialogue requires more than producing a factual rebuttal: users differ in what they know, what they believe, and what they need to hear, so an e…
Palette: A Modular, Controllable, and Efficient Framework for On-demand Authorized Safety Alignment Relaxation in LLMs
Qitao Tan, Xiaoying Song, Arman Akbari +7
Current safety alignment of foundation models largely follows a \emph{one-size-fits-all} paradigm, applying the same refusal policy across users and contexts. As a result, models m…
ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Models
Arash Akbari, Arman Akbari, Masih Eskandar +11
Vision-Language-Action (VLA) models exhibit remarkable action generation for embodied intelligence, but their heavy compute make deployment on edge platforms impractical. Aggressiv…
Q-realign: Piggybacking Realignment on Quantization for Safe and Efficient LLM Deployment
Qitao Tan, Xiaoying Song, Ningxi Cheng +6
Public large language models (LLMs) are typically safety-aligned during pretraining, yet task-specific fine-tuning required for deployment often erodes this alignment and introduce…
Towards Fast LLM Fine-tuning through Zeroth-Order Optimization with Projected Gradient-Aligned Perturbations
Zhendong Mi, Qitao Tan, Grace Li Zhang +3
Fine-tuning large language models (LLMs) using zeroth-order (ZO) optimization has emerged as a promising alternative to traditional gradient-based methods due to its reduced memory…
End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost
Qitao Tan, Xiaoying Song, Jin Lu +9
Quantization is an effective technique to reduce the deployment cost of large language models (LLMs), and post-training quantization (PTQ) has been widely studied due to its effici…
Rethinking the Potential of Layer Freezing for Efficient DNN Training
Chence Yang, Ci Zhang, Lei Lu +11
With the growing size of deep neural networks and datasets, the computational costs of training have significantly increased. The layer-freezing technique has recently attracted gr…
KerZOO: Kernel Function Informed Zeroth-Order Optimization for Accurate and Accelerated LLM Fine-Tuning
Zhendong Mi, Qitao Tan, Xiaodong Yu +3
Large language models (LLMs) have demonstrated impressive capabilities across numerous NLP tasks. Nevertheless, conventional first-order fine-tuning techniques impose heavy memory…
Perturbation-efficient Zeroth-order Optimization for Hardware-friendly On-device Training
Qitao Tan, Sung-En Chang, Rui Xia +10
Zeroth-order (ZO) optimization is an emerging deep neural network (DNN) training paradigm that offers computational simplicity and memory savings. However, this seemingly promising…
Harmony in Divergence: Towards Fast, Accurate, and Memory-efficient Zeroth-order LLM Fine-tuning
Qitao Tan, Jun Liu, Zheng Zhan +6
Large language models (LLMs) excel across various tasks, but standard first-order (FO) fine-tuning demands considerable memory, significantly limiting real-world deployment. Recent…