247 citations · 400 across the 46 of their papers we have counts for
10 papers · 1 filter
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way
Kai Lv, Shuo Zhang, Tianle Gu +11
Large language models (LLMs) are increasingly pivotal in a wide range of natural language processing tasks. Access to pre-trained models, courtesy of the open-source community, has…
Watermarking LLMs with Weight Quantization
Linyang Li, Botian Jiang, Pengyu Wang +3
Abuse of large language models reveals high risks as large language models are being deployed at an astonishing speed. It is important to protect the model weights to avoid malicio…
AdaLomo: Low-memory Optimization with Adaptive Learning Rate
Kai Lv, Hang Yan, Qipeng Guo +2
Large language models have achieved remarkable success, but their extensive parameter size necessitates substantial memory for training, thereby setting a high threshold. While the…
Scaling Laws of RoPE-based Extrapolation
Xiaoran Liu, Hang Yan, Shuo Zhang +3
The extrapolation capability of Large Language Models (LLMs) based on Rotary Position Embedding is currently a topic of considerable interest. The mainstream approach to addressing…
WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Models
Conghui He, Zhenjiang Jin, Chao Xu +6
The rise in popularity of ChatGPT and GPT-4 has significantly accelerated the development of large models, leading to the creation of numerous impressive large language models(LLMs…
Does Correction Remain A Problem For Large Language Models?
Xiaowu Zhang, Xiaotian Zhang, Cheng Yang +2
As large language models, such as GPT, continue to advance the capabilities of natural language processing (NLP), the question arises: does the problem of correction still persist?…
Secrets of RLHF in Large Language Models Part I: PPO
Rui Zheng, Shihan Dou, Songyang Gao +24
Large language models (LLMs) have formulated a blueprint for the advancement of artificial general intelligence. Its primary objective is to function as a human-centric (helpful, h…
PromptNER: A Prompting Method for Few-shot Named Entity Recognition via k Nearest Neighbor Search
Mozhi Zhang, Hang Yan, Yaqian Zhou +1
Few-shot Named Entity Recognition (NER) is a task aiming to identify named entities via limited annotated samples. Recently, prototypical networks have shown promising performance…
Unified Demonstration Retriever for In-Context Learning
Xiaonan Li, Kai Lv, Hang Yan +6
In-context learning is a new learning paradigm where a language model conditions on a few input-output pairs (demonstrations) and a test input, and directly outputs the prediction.…
CodeIE: Large Code Generation Models are Better Few-Shot Information Extractors
Peng Li, Tianxiang Sun, Qiong Tang +4
Large language models (LLMs) pre-trained on massive corpora have demonstrated impressive few-shot learning ability on many NLP tasks. A common practice is to recast the task into a…