19 citations · 19 across the 4 of their papers we have counts for
4 papers · 1 filter
Shuffle the Context: RoPE-Perturbed Self-Distillation for Long-Context Adaptation
Zichong Li, Chen Liang, Liliang Ren +3
Large language models (LLMs) increasingly operate in settings that require reliable long-context understanding, such as retrieval-augmented generation and multi-document reasoning.…
LLMs Can Generate a Better Answer by Aggregating Their Own Responses
Zichong Li, Xinyu Feng, Yuheng Cai +6
Large Language Models (LLMs) have shown remarkable capabilities across tasks, yet they often require additional prompting techniques when facing complex problems. While approaches…
LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models
Yixiao Li, Yifan Yu, Chen Liang +4
Quantization is an indispensable technique for serving Large Language Models (LLMs) and has recently found its way into LoRA fine-tuning. In this work we focus on the scenario wher…
Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective
Ming Zhong, Chenxin An, Weizhu Chen +2
Large Language Models (LLMs) inherently encode a wealth of knowledge within their parameters through pre-training on extensive corpora. While prior research has delved into operati…