1.5k citations · 5.2k across the 127 of their papers we have counts for
14 papers · 1 filter
AdaLomo: Low-memory Optimization with Adaptive Learning Rate
Kai Lv, Hang Yan, Qipeng Guo +2
Large language models have achieved remarkable success, but their extensive parameter size necessitates substantial memory for training, thereby setting a high threshold. While the…
Character-LLM: A Trainable Agent for Role-Playing
Yunfan Shao, Linyang Li, Junqi Dai +1
Large language models (LLMs) can be used to serve as agents to simulate human behaviors, given the powerful ability to understand human instructions and provide high-quality genera…
Scaling Laws of RoPE-based Extrapolation
Xiaoran Liu, Hang Yan, Shuo Zhang +3
The extrapolation capability of Large Language Models (LLMs) based on Rotary Position Embedding is currently a topic of considerable interest. The mainstream approach to addressing…
The Rise and Potential of Large Language Model Based Agents: A Survey
Zhiheng Xi, Wenxiang Chen, Xin Guo +26
For a long time, humanity has pursued artificial intelligence (AI) equivalent to or surpassing the human level, with AI agents considered a promising vehicle for this pursuit. AI a…
SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models
Xin Zhang, Dong Zhang, Shimin Li +2
Current speech large language models build upon discrete speech representations, which can be categorized into semantic tokens and acoustic tokens. However, existing speech tokens…
Secrets of RLHF in Large Language Models Part I: PPO
Rui Zheng, Shihan Dou, Songyang Gao +24
Large language models (LLMs) have formulated a blueprint for the advancement of artificial general intelligence. Its primary objective is to function as a human-centric (helpful, h…
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
Chenxin An, Shansan Gong, Ming Zhong +5
Recently, there has been growing interest in extending the context length of large language models (LLMs), aiming to effectively process long inputs of one turn or conversations wi…
Distributed Marker Representation for Ambiguous Discourse Markers and Entangled Relations
Dongyu Ru, Lin Qiu, Xipeng Qiu +2
Discourse analysis is an important task because it models intrinsic semantic structures between sentences in a document. Discourse markers are natural representations of discourse…
From Hypergraph Energy Functions to Hypergraph Neural Networks
Yuxin Wang, Quan Gan, Xipeng Qiu +2
Hypergraphs are a powerful abstraction for representing higher-order interactions between entities of interest. To exploit these relationships in making downstream predictions, a v…
Full Parameter Fine-tuning for Large Language Models with Limited Resources
Kai Lv, Yuqing Yang, Tengxiao Liu +3
Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) but demand massive GPU resources for training. Lowering the threshold for LLMs training would enc…
An AMR-based Link Prediction Approach for Document-level Event Argument Extraction
Yuqing Yang, Qipeng Guo, Xiangkun Hu +3
Recent works have introduced Abstract Meaning Representation (AMR) for Document-level Event Argument Extraction (Doc-level EAE), since AMR provides a useful interpretation of compl…
Do Large Language Models Know What They Don't Know?
Zhangyue Yin, Qiushi Sun, Qipeng Guo +3
Large language models (LLMs) have a wealth of knowledge that allows them to excel in various Natural Language Processing (NLP) tasks. Current research focuses on enhancing their pe…
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities
Dong Zhang, Shimin Li, Xin Zhang +4
Multi-modal large language models are regarded as a crucial step towards Artificial General Intelligence (AGI) and have garnered significant interest with the emergence of ChatGPT.…
Unified Demonstration Retriever for In-Context Learning
Xiaonan Li, Kai Lv, Hang Yan +6
In-context learning is a new learning paradigm where a language model conditions on a few input-output pairs (demonstrations) and a test input, and directly outputs the prediction.…