256 citations · 742 across the 137 of their papers we have counts for
22 papers · 2 filters
LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
Shihan Dou, Enyu Zhou, Yan Liu +13
Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks.…
Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback
Wei Shen, Rui Zheng, Wenyu Zhan +5
Reinforcement learning from human feedback serves as a crucial bridge, aligning large language models with human and societal values. This alignment requires a vast corpus of human…
Rescue: Ranking LLM Responses with Partial Ordering to Improve Response Generation
Yikun Wang, Rui Zheng, Haoming Li +3
Customizing LLMs for a specific task involves separating high-quality responses from lower-quality ones. This skill can be developed using supervised fine-tuning with extensive hum…
Unveiling A Core Linguistic Region in Large Language Models
Jun Zhao, Zhihao Zhang, Yide Ma +4
Brain localization, which describes the association between specific regions of the brain and their corresponding functions, is widely accepted in the field of cognitive science as…
Orthogonal Subspace Learning for Language Model Continual Learning
Xiao Wang, Tianze Chen, Qiming Ge +6
Benefiting from massive corpora and advanced hardware, large language models (LLMs) exhibit remarkable capabilities in language understanding and generation. However, their perform…
RealBehavior: A Framework for Faithfully Characterizing Foundation Models' Human-like Behavior Mechanisms
Enyu Zhou, Rui Zheng, Zhiheng Xi +7
Reports of human-like behaviors in foundation models are growing, with psychological theories providing enduring tools to investigate these behaviors. However, current research ten…
Reading Order Matters: Information Extraction from Visually-rich Documents by Token Path Prediction
Chong Zhang, Ya Guo, Yi Tu +5
Recent advances in multimodal pre-trained models have significantly improved information extraction from visually-rich documents (VrDs), in which named entity recognition (NER) is…
Universal Multi-modal Entity Alignment via Iteratively Fusing Modality Similarity Paths
Bolin Zhu, Xiaoze Liu, Xin Mao +4
The objective of Entity Alignment (EA) is to identify equivalent entity pairs from multiple Knowledge Graphs (KGs) and create a more comprehensive and unified KG. The majority of E…
TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models
Xiao Wang, Yuansen Zhang, Tianze Chen +9
Aligned large language models (LLMs) demonstrate exceptional capabilities in task-solving, following instructions, and ensuring safety. However, the continual learning aspect of th…
RethinkingTMSC: An Empirical Study for Target-Oriented Multimodal Sentiment Classification
Junjie Ye, Jie Zhou, Junfeng Tian +4
Recently, Target-oriented Multimodal Sentiment Classification (TMSC) has gained significant attention among scholars. However, current multimodal models have reached a performance…
Secrets of RLHF in Large Language Models Part I: PPO
Rui Zheng, Shihan Dou, Songyang Gao +24
Large language models (LLMs) have formulated a blueprint for the advancement of artificial general intelligence. Its primary objective is to function as a human-centric (helpful, h…
Actively Supervised Clustering for Open Relation Extraction
Jun Zhao, Yongxin Zhang, Qi Zhang +4
Current clustering-based Open Relation Extraction (OpenRE) methods usually adopt a two-stage pipeline. The first stage simultaneously learns relation representations and assignment…
RE-Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction
Jun Zhao, Wenyu Zhan, Xin Zhao +6
Semantic matching is a mainstream paradigm of zero-shot relation extraction, which matches a given input with a corresponding label description. The entities in the input should ex…
Open Set Relation Extraction via Unknown-Aware Training
Jun Zhao, Xin Zhao, Wenyu Zhan +6
The existing supervised relation extraction methods have achieved impressive performance in a closed-set setting, where the relations during both training and testing remain the sa…
Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model
Xiao Wang, Weikang Zhou, Qi Zhang +7
Pretrained language models have achieved remarkable success in various natural language processing tasks. However, pretraining has recently shifted toward larger models and larger…
Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization
Ting Wu, Rui Zheng, Tao Gui +2
Models trained with empirical risk minimization (ERM) are revealed to easily rely on spurious correlations, resulting in poor generalization. Group distributionally robust optimiza…
Enhancing Contrastive Learning with Noise-Guided Attack: Towards Continual Relation Extraction in the Wild
Ting Wu, Jingyi Liu, Rui Zheng +3
The principle of continual relation extraction~(CRE) involves adapting to emerging novel relations while preserving od knowledge. While current endeavors in CRE succeed in preservi…
Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement
Zhiheng Xi, Senjie Jin, Yuhao Zhou +5
To enhance the multi-step reasoning capabilities of large language models, researchers have extensively explored prompting methods, notably the Chain-of-Thought (CoT) method which…
A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition
Limao Xiong, Jie Zhou, Qunxi Zhu +7
Existing models for named entity recognition (NER) are mainly based on large-scale labeled datasets, which always obtain using crowdsourcing. However, it is hard to obtain a unifie…
InstructUIE: Multi-task Instruction Tuning for Unified Information Extraction
Xiao Wang, Weikang Zhou, Can Zu +11
Large language models have unlocked strong multi-task capabilities from reading instructive prompts. However, recent studies have shown that existing large models still have diffic…
A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models
Junjie Ye, Xuanting Chen, Nuo Xu +12
GPT series models, such as GPT-3, CodeX, InstructGPT, ChatGPT, and so on, have gained considerable attention due to their exceptional natural language processing capabilities. Howe…
How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks
Xuanting Chen, Junjie Ye, Can Zu +7
The GPT-3.5 models have demonstrated impressive performance in various Natural Language Processing (NLP) tasks, showcasing their strong understanding and reasoning capabilities. Ho…