4 citations · 7 across the 7 of their papers we have counts for
8 papers
LoRAQuant: Mixed-Precision Quantization of LoRA to Ultra-Low Bits
Amir Reza Mirzaei, Yuqiao Wen, Yanshuai Cao +1
Low-Rank Adaptation (LoRA) has become a popular technique for parameter-efficient fine-tuning of large language models (LLMs). In many real-world scenarios, multiple adapters are l…
Exploring Model Invariance with Discrete Search for Ultra-Low-Bit Quantization
Yuqiao Wen, Yanshuai Cao, Lili Mou
Large language models have been increasing in size due to their success in a wide range of applications. This calls for a pressing need to reduce memory usage to make them more acc…
Revisiting Intermediate-Layer Matching in Knowledge Distillation: Layer-Selection Strategy Doesn't Matter (Much)
Zony Yu, Yuqiao Wen, Lili Mou
Knowledge distillation (KD) is a popular method of transferring knowledge from a large "teacher" model to a small "student" model. Previous work has explored various layer-selectio…
EBBS: An Ensemble with Bi-Level Beam Search for Zero-Shot Machine Translation
Yuqiao Wen, Behzad Shayegh, Chenyang Huang +2
The ability of zero-shot translation emerges when we train a multilingual model with certain translation directions; the model can then directly translate in unseen directions. Alt…
Tree-Averaging Algorithms for Ensemble-Based Unsupervised Discontinuous Constituency Parsing
Behzad Shayegh, Yuqiao Wen, Lili Mou
We address unsupervised discontinuous constituency parsing, where we observe a high variance in the performance of the only previous model in the literature. We propose to build an…
f-Divergence Minimization for Sequence-Level Knowledge Distillation
Yuqiao Wen, Zichao Li, Wenyu Du +1
Knowledge distillation (KD) is the process of transferring knowledge from a large model to a small one. It has gained increasing attention in the natural language processing commun…
An Equal-Size Hard EM Algorithm for Diverse Dialogue Generation
Yuqiao Wen, Yongchang Hao, Yanshuai Cao +1
Open-domain dialogue systems aim to interact with humans through natural language texts in an open-ended fashion. Despite the recent success of super large dialogue systems such as…
An Empirical Study on the Overlapping Problem of Open-Domain Dialogue Datasets
Yuqiao Wen, Guoqing Luo, Lili Mou
Open-domain dialogue systems aim to converse with humans through text, and dialogue research has heavily relied on benchmark datasets. In this work, we observe the overlapping prob…