17 citations · 48 across the 13 of their papers we have counts for
6 papers · 1 filter
PACIT: Unlocking the Power of Examples for Better In-Context Instruction Tuning
Tianci Xue, Ziqi Wang, Yixia Li +2
Instruction tuning enhances the instruction following ability of large language models by finetuning with supervised instruction data. Previous work proposes in-context instruction…
Enabling Language Models to Implicitly Learn Self-Improvement
Ziqi Wang, Le Hou, Tianjian Lu +4
Large Language Models (LLMs) have demonstrated remarkable capabilities in open-ended text generation tasks. However, the inherent open-ended nature of these tasks implies that ther…
Parameter-Efficient Tuning Helps Language Model Alignment
Tianci Xue, Ziqi Wang, Heng Ji
Aligning large language models (LLMs) with human preferences is essential for safe and useful LLMs. Previous works mainly adopt reinforcement learning (RLHF) and direct preference…
Understanding the Effect of Data Augmentation on Knowledge Distillation
Ziqi Wang, Chi Han, Wenxuan Bao +1
Knowledge distillation (KD) requires sufficient data to transfer knowledge from large-scale teacher models to small-scale student models. Therefore, data augmentation has been wide…
Understanding Emergent In-Context Learning from a Kernel Regression Perspective
Chi Han, Ziqi Wang, Han Zhao +1
Large language models (LLMs) have initiated a paradigm shift in transfer learning. In contrast to the classic pretraining-then-finetuning procedure, in order to use LLMs for downst…
RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thought
Tianci Xue, Ziqi Wang, Zhenhailong Wang +3
Large language Models (LLMs) have achieved promising performance on arithmetic reasoning tasks by incorporating step-by-step chain-of-thought (CoT) prompting. However, LLMs face ch…