5 papers
Break Through the Compression Bottleneck: From Theory to Practice
Xiusheng Huang, Lu Wang, Yequan Wang +2
As the parameter size of language models continues to grow, effective model compression is required to reduce their computational and memory overhead. Existing compression methods…
Theory-optimal Quantization Based on Flatness
Xiusheng Huang, Zhe Li, Xuanwu Yin +5
Post-training quantization has emerged as a widely adopted technique for compressing and accelerating the inference of Large Language Models (LLMs). The primary challenges in LLMs…
Linear-Time Demonstration Selection for In-Context Learning via Gradient Estimation
Ziniu Zhang, Zhenshuo Zhang, Dongyue Li +3
This paper introduces an algorithm to select demonstration examples for in-context learning of a query set. Given a set of examples, how can we quickly select out of to…
Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach
Dongyue Li, Ziniu Zhang, Lu Wang +1
We study the problem of fine-tuning a language model (LM) for a target task by optimally using the information from auxiliary tasks. This problem has broad applications in NLP,…
Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets
Dongyue Li, Ziniu Zhang, Lu Wang +1
This paper develops an ensemble method for fine-tuning a language model to multiple datasets. Existing methods, such as quantized LoRA (QLoRA), are efficient when adapting to a sin…