256 citations · 349 across the 25 of their papers we have counts for
10 papers · 1 filter
LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
Shihan Dou, Enyu Zhou, Yan Liu +13
Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks.…
Orthogonal Subspace Learning for Language Model Continual Learning
Xiao Wang, Tianze Chen, Qiming Ge +6
Benefiting from massive corpora and advanced hardware, large language models (LLMs) exhibit remarkable capabilities in language understanding and generation. However, their perform…
TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
Xiao Wang, Yuansen Zhang, Tianze Chen +9
Aligned large language models (LLMs) demonstrate exceptional capabilities in task-solving, following instructions, and ensuring safety. However, the continual learning aspect of th…
Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models
Xianjun Yang, Xiao Wang, Qi Zhang +4
Warning: This paper contains examples of harmful language, and reader discretion is recommended. The increasing open release of powerful large language models (LLMs) has facilitate…
Improving Generalization of Alignment with Human Preferences through Group Invariant Learning
Rui Zheng, Wei Shen, Yuan Hua +9
The success of AI assistants based on language models (LLMs) hinges crucially on Reinforcement Learning from Human Feedback (RLHF), which enables the generation of responses more a…
The Rise and Potential of Large Language Model Based Agents: A Survey
Zhiheng Xi, Wenxiang Chen, Xin Guo +26
For a long time, humanity has pursued artificial intelligence (AI) equivalent to or surpassing the human level, with AI agents considered a promising vehicle for this pursuit. AI a…
DSRM: Boost Textual Adversarial Training with Distribution Shift Risk Minimization
Songyang Gao, Shihan Dou, Yan Liu +5
Adversarial training is one of the best-performing methods in improving the robustness of deep language models. However, robust models come at the cost of high time consumption, as…
Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model
Xiao Wang, Weikang Zhou, Qi Zhang +7
Pretrained language models have achieved remarkable success in various natural language processing tasks. However, pretraining has recently shifted toward larger models and larger…
A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition
Limao Xiong, Jie Zhou, Qunxi Zhu +7
Existing models for named entity recognition (NER) are mainly based on large-scale labeled datasets, which always obtain using crowdsourcing. However, it is hard to obtain a unifie…
InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction
Xiao Wang, Weikang Zhou, Can Zu +11
Large language models have unlocked strong multi-task capabilities from reading instructive prompts. However, recent studies have shown that existing large models still have diffic…