6 papers · 1 filter
Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
Chenghao Fan, Zhenyi Lu, Sichen Liu +4
While Low-Rank Adaptation (LoRA) enables parameter-efficient fine-tuning for Large Language Models (LLMs), its performance often falls short of Full Fine-Tuning (Full FT). Current…
Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models
Tingchen Fu, Jiawei Gu, Yafu Li +2
Instruction-following is essential for aligning large language models (LLMs) with user intent. While recent reasoning-oriented models exhibit impressive performance on complex math…
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion
Chenghao Fan, Zhenyi Lu, Wei Wei +4
Efficient fine-tuning of large language models for task-specific applications is imperative, yet the vast number of parameters in these models makes their training increasingly cha…
Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
Zhenyi Lu, Chenghao Fan, Wei Wei +3
In the era of large language models, model merging is a promising way to combine multiple task-specific models into a single multitask model without extra training. However, two ch…
Mitigating Boundary Ambiguity and Inherent Bias for Text Classification in the Era of Large Language Models
Zhenyi Lu, Jie Tian, Wei Wei +4
Text classification is a crucial task encountered frequently in practical scenarios, yet it is still under-explored in the era of large language models (LLMs). This study shows tha…
Enhancing Low-Resource Relation Representations through Multi-View Decoupling
Chenghao Fan, Wei Wei, Xiaoye Qu +4
Recently, prompt-tuning with pre-trained language models (PLMs) has demonstrated the significantly enhancing ability of relation extraction (RE) tasks. However, in low-resource sce…