10 papers · 1 filter
AlignDiff: Exploiting Model-Intrinsic Information for Better Preference Data Selection
Peng Lai, He Zhu, Zhiwen Ruan +6
Aligning large language models with human preferences remains a challenge, primarily due to the critical role of preference data quality in effective alignment. Existing datasets a…
VFA: Empowering Multilingual MLLMs via Vision-Free Adaptation
Yixia Li, Yaqing Shi, Zhiwen Ruan +6
Multimodal large language models have advanced rapidly, yet most remain English-centric, as scaling multilingual multimodal instruction tuning is limited by the scarcity and high c…
Bridging the Agent-World Gap: Text World Models for LLM-based Agents
Yixia Li, Hongru Wang, Peng Lai +13
Large language model (LLM)-based agents are increasingly used in interactive textual environments, from web navigation and code editing to tool use and long-horizon dialogue. Yet m…
GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models
Zhiwen Ruan, Yichao Du, Jianjie Zheng +6
A promising paradigm for adapting instruction-tuned language models is to learn task-specific updates on a pretrained base model and subsequently merge them into the instruction-tu…
G2: Guided Generation for Enhanced Output Diversity in LLMs
Zhiwen Ruan, Yixia Li, Yefeng Liu +5
Large Language Models (LLMs) have demonstrated exceptional performance across diverse natural language processing tasks. However, these models exhibit a critical limitation in outp…
Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
Zhiwen Ruan, Yixia Li, He Zhu +4
Large language models (LLMs) primarily rely on supervised fine-tuning (SFT) as a key method to adapt pre-trained models to domain-specific tasks such as mathematical reasoning. How…
Unveiling Over-Memorization in Finetuning LLMs for Reasoning Tasks
Zhiwen Ruan, Yun Chen, Yutao Hou +3
The pretrained large language models (LLMs) are finetuned with labeled data for better instruction following ability and alignment with human values. In this paper, we study the le…
TAG-INSTRUCT: Controlled Instruction Complexity Enhancement through Structure-based Augmentation
He Zhu, Zhiwen Ruan, Junyou Su +4
High-quality instruction data is crucial for developing large language models (LLMs), yet existing approaches struggle to effectively control instruction complexity. We present TAG…
LayAlign: Enhancing Multilingual Reasoning in Large Language Models via Layer-Wise Adaptive Fusion and Alignment Strategy
Zhiwen Ruan, Yixia Li, He Zhu +5
Despite being pretrained on multilingual corpora, large language models (LLMs) exhibit suboptimal performance on low-resource languages. Recent approaches have leveraged multilingu…
Compound-QA: A Benchmark for Evaluating LLMs on Compound Questions
Yutao Hou, Yajing Luo, Zhiwen Ruan +4
Large language models (LLMs) demonstrate remarkable performance across various tasks, prompting researchers to develop diverse evaluation benchmarks. However, most benchmarks typic…