1.6k citations · 1.9k across the 43 of their papers we have counts for
44 papers
Exploring Backdoor Vulnerabilities of Chat Models
Yunzhuo Hao, Wenkai Yang, Yankai Lin
Recent researches have shown that Large Language Models (LLMs) are susceptible to a security threat known as Backdoor Attack. The backdoored model will behave well in normal cases…
Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents
Wenkai Yang, Xiaohan Bi, Yankai Lin +3
Driven by the rapid development of Large Language Models (LLMs), LLM-based agents have been developed to handle various real-world applications, including finance, healthcare, and…
MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation
Xiaozhi Wang, Hao Peng, Yong Guan +9
Understanding events in texts is a core objective of natural language understanding, which requires detecting event occurrences, extracting event arguments, and analyzing inter-eve…
Distilling Rule-based Knowledge into Large Language Models
Wenkai Yang, Yankai Lin, Jie Zhou +1
Large language models (LLMs) have shown incredible performance in completing various real-world tasks. The current paradigm of knowledge learning for LLMs is mainly based on learni…
Rational Decision-Making Agent with Internalized Utility Judgment
Yining Ye, Xin Cong, Shizuo Tian +5
Large language models (LLMs) have demonstrated remarkable advancements and have attracted significant efforts to develop LLMs into agents capable of executing intricate multi-step…
A Survey on Large Language Model based Autonomous Agents
Lei Wang, Chen Ma, Xueyang Feng +10
Autonomous agents have long been a prominent research focus in both academic and industry communities. Previous research in this field often focuses on training agents with limited…
ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
Yujia Qin, Shihao Liang, Yining Ye +16
Despite the advancements of open-source large language models (LLMs), e.g., LLaMA, they remain significantly limited in tool-use capabilities, i.e., using external tools (APIs) to…
Towards Codable Watermarking for Injecting Multi-bits Information to LLMs
Lean Wang, Wenkai Yang, Deli Chen +5
As large language models (LLMs) generate texts with increasing fluency and realism, there is a growing need to identify the source of texts to prevent the abuse of LLMs. Text water…
User Behavior Simulation with Large Language Model based Agents
Lei Wang, Jingsen Zhang, Hao Yang +11
Simulating high quality user behavior data has always been a fundamental problem in human-centered applications, where the major difficulty originates from the intricate mechanism…
Exploring the Impact of Model Scaling on Parameter-Efficient Tuning
Yusheng Su, Chi-Min Chan, Jiali Cheng +9
Parameter-efficient tuning (PET) methods can effectively drive extremely large pre-trained language models (PLMs) by training only minimal parameters. Different PET methods utilize…
Stochastic Bridges as Effective Regularizers for Parameter-Efficient Tuning
Weize Chen, Xu Han, Yankai Lin +3
Parameter-efficient tuning methods (PETs) have achieved promising results in tuning large pre-trained language models (PLMs). By formalizing frozen PLMs and additional tunable para…
WebCPM: Interactive Web Search for Chinese Long-form Question Answering
Yujia Qin, Zihan Cai, Dian Jin +12
Long-form question answering (LFQA) aims at answering complex, open-ended questions with detailed, paragraph-length responses. The de facto paradigm of LFQA necessitates two proced…
Emergent Modularity in Pre-trained Transformers
Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin +7
This work examines the presence of modularity in pre-trained Transformers, a feature commonly found in human brains and thought to be vital for general intelligence. In analogy to…
Plug-and-Play Knowledge Injection for Pre-trained Language Models
Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin +8
Injecting external knowledge can improve the performance of pre-trained language models (PLMs) on various downstream NLP tasks. However, massive retraining is required to deploy ne…
Tool Learning with Foundation Models
Yujia Qin, Shengding Hu, Yankai Lin +38
Humans possess an extraordinary ability to create and utilize tools, allowing them to overcome physical limitations and explore new frontiers. With the advent of foundation models,…
When to Trust Aggregated Gradients: Addressing Negative Client Sampling in Federated Learning
Wenkai Yang, Yankai Lin, Guangxiang Zhao +3
Federated Learning has become a widely-used framework which allows learning a global model on decentralized local datasets under the condition of protecting local data privacy. How…
MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction
Xiaozhi Wang, Yulin Chen, Ning Ding +9
The diverse relationships among real-world events, including coreference, temporal, causal, and subevent relations, are fundamental to understanding natural languages. However, two…
Exploring Mode Connectivity for Pre-trained Language Models
Yujia Qin, Cheng Qian, Jing Yi +6
Recent years have witnessed the prevalent application of pre-trained language models (PLMs) in NLP. From the perspective of parameter space, PLMs provide generic initialization, st…
Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning
Jing Yi, Weize Chen, Yujia Qin +6
Delta tuning (DET, also known as parameter-efficient tuning) is deemed as the new paradigm for using pre-trained language models (PLMs). Up to now, various DETs with distinct desig…
ROSE: Robust Selective Fine-tuning for Pre-trained Language Models
Lan Jiang, Hao Zhou, Yankai Lin +3
Even though the large-scale language models have achieved excellent performances, they suffer from various adversarial attacks. A large body of defense methods has been proposed. H…
From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models
Lei Li, Yankai Lin, Xuancheng Ren +4
Investigating better ways to reuse the released pre-trained language models (PLMs) can significantly reduce the computational cost and the potential environmental side-effects. Thi…
Automatic Label Sequence Generation for Prompting Sequence-to-sequence Models
Zichun Yu, Tianyu Gao, Zhengyan Zhang +4
Prompting, which casts downstream applications as language modeling tasks, has shown to be sample efficient compared to standard fine-tuning with pre-trained models. However, one p…
Manual-Guided Dialogue for Flexible Conversational Agents
Ryuichi Takanobu, Hao Zhou, Yankai Lin +3
How to build and use dialogue data efficiently, and how to deploy models in different domains at scale can be two critical issues in building a task-oriented dialogue system. In th…
Towards a General Pre-training Framework for Adaptive Learning in MOOCs
Qingyang Zhong, Jifan Yu, Zheyuan Zhang +6
Adaptive learning aims to stimulate and meet the needs of individual learners, which requires sophisticated system-level coordination of diverse tasks, including modeling learning…