3 papers
cs.LG2025
Linear-Time Demonstration Selection for In-Context Learning via Gradient Estimation
Ziniu Zhang, Zhenshuo Zhang, Dongyue Li +3
This paper introduces an algorithm to select demonstration examples for in-context learning of a query set. Given a set of examples, how can we quickly select out of to…
cs.LG2025
Efficient Ensemble for Fine-tuning Language Models on Multiple Datasets
Dongyue Li, Ziniu Zhang, Lu Wang +1
This paper develops an ensemble method for fine-tuning a language model to multiple datasets. Existing methods, such as quantized LoRA (QLoRA), are efficient when adapting to a sin…
cs.CL2024
Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach
Dongyue Li, Ziniu Zhang, Lu Wang +1
We study the problem of fine-tuning a language model (LM) for a target task by optimally using the information from auxiliary tasks. This problem has broad applications in NLP,…