67 citations · 245 across the 29 of their papers we have counts for
31 papers
GAOKAO-Eval: Does high scores truly reflect strong capabilities in LLMs?
Zhikai Lei, Tianyi Liang, Hanglei Hu +8
Large Language Models (LLMs) are commonly evaluated using human-crafted benchmarks, under the premise that higher scores implicitly reflect stronger human-like performance. However…
Unified Active Retrieval for Retrieval Augmented Generation
Qinyuan Cheng, Xiaonan Li, Shimin Li +7
In Retrieval-Augmented Generation (RAG), retrieval is not always helpful and applying it to every instruction is sub-optimal. Therefore, determining whether to retrieve is crucial…
InternLM2 Technical Report
Zheng Cai, Maosong Cao, Haojiong Chen +97
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI). However, replicating such advan…
Turn Waste into Worth: Rectifying Top- Router of MoE
Zhiyuan Zeng, Qipeng Guo, Zhaoye Fei +7
Sparse Mixture of Experts (MoE) models are popular for training large language models due to their computational efficiency. However, the commonly used top- routing mechanism su…
Balanced Data Sampling for Language Model Training with Clustering
Yunfan Shao, Linyang Li, Zhaoye Fei +3
Data plays a fundamental role in the training of Large Language Models (LLMs). While attention has been paid to the collection and composition of datasets, determining the data sam…
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling
Jun Zhan, Junqi Dai, Jiasheng Ye +13
We introduce AnyGPT, an any-to-any multimodal language model that utilizes discrete representations for the unified processing of various modalities, including speech, text, images…
InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning
Huaiyuan Ying, Shuo Zhang, Linyang Li +19
The math abilities of large language models can represent their abstract reasoning ability. In this paper, we introduce and open-source our math reasoning LLMs InternLM-Math which…
Can AI Assistants Know What They Don't Know?
Qinyuan Cheng, Tianxiang Sun, Xiangyang Liu +7
Recently, AI assistants based on large language models (LLMs) show surprising performance in many tasks, such as dialogue, solving math problems, writing code, and using tools. Alt…
InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance
Pengyu Wang, Dong Zhang, Linyang Li +5
With the rapid development of large language models (LLMs), they are not only used as general-purpose AI assistants but are also customized through further fine-tuning to meet the…
Unearthing Large Scale Domain-Specific Knowledge from Public Corpora
Zhaoye Fei, Yunfan Shao, Linyang Li +6
Large language models (LLMs) have demonstrated remarkable potential in various tasks, however, there remains a significant lack of open-source models and data for specific domains.…
LLatrieval: LLM-Verified Retrieval for Verifiable Generation
Xiaonan Li, Changtai Zhu, Linyang Li +3
Verifiable generation aims to let the large language model (LLM) generate text with supporting documents, which enables the user to flexibly verify the answer and makes the LLM's o…
Watermarking LLMs with Weight Quantization
Linyang Li, Botian Jiang, Pengyu Wang +3
Abuse of large language models reveals high risks as large language models are being deployed at an astonishing speed. It is important to protect the model weights to avoid malicio…
PerturbScore: Connecting Discrete and Continuous Perturbations in NLP
Linyang Li, Ke Ren, Yunfan Shao +2
With the rapid development of neural network applications in NLP, model robustness problem is gaining more attention. Different from computer vision, the discrete nature of texts m…
Character-LLM: A Trainable Agent for Role-Playing
Yunfan Shao, Linyang Li, Junqi Dai +1
Large language models (LLMs) can be used to serve as agents to simulate human behaviors, given the powerful ability to understand human instructions and provide high-quality genera…
SeqXGPT: Sentence-Level AI-Generated Text Detection
Pengyu Wang, Linyang Li, Ke Ren +3
Widely applied large language models (LLMs) can generate human-like content, raising concerns about the abuse of LLMs. Therefore, it is important to build strong AI-generated text…
Multijugate Dual Learning for Low-Resource Task-Oriented Dialogue System
Shimin Li, Xiaotian Zhang, Yanjun Zheng +2
Dialogue data in real scenarios tend to be sparsely available, rendering data-starved end-to-end dialogue systems trained inadequately. We discover that data utilization efficiency…
Improving Contrastive Learning of Sentence Embeddings from AI Feedback
Qinyuan Cheng, Xiaogui Yang, Tianxiang Sun +2
Contrastive learning has become a popular approach in natural language processing, particularly for the learning of sentence embeddings. However, the discrete nature of natural lan…
Origin Tracing and Detecting of LLMs
Linyang Li, Pengyu Wang, Ke Ren +2
The extraordinary performance of large language models (LLMs) heightens the importance of detecting whether the context is generated by an AI system. More importantly, while more a…
Mitigating Negative Style Transfer in Hybrid Dialogue System
Shimin Li, Qinyuan Cheng, Linyang Li +1
As the functionality of dialogue systems evolves, hybrid dialogue systems that accomplish user-specific goals and participate in open-topic chitchat with users are attracting growi…
Is MultiWOZ a Solved Task? An Interactive TOD Evaluation Framework with User Simulator
Qinyuan Cheng, Linyang Li, Guofeng Quan +3
Task-Oriented Dialogue (TOD) systems are drawing more and more attention in recent studies. Current methods focus on constructing pre-trained models or fine-tuning strategies while…
"Is Whole Word Masking Always Better for Chinese BERT?": Probing on Chinese Grammatical Error Correction
Yong Dai, Linyang Li, Cong Zhou +5
Whole word masking (WWM), which masks all subwords corresponding to a word at once, makes a better English BERT model. For the Chinese language, however, there is no subword becaus…
Text Adversarial Purification as Defense against Adversarial Attacks
Linyang Li, Demin Song, Xipeng Qiu
Adversarial purification is a successful defense mechanism against adversarial attacks without requiring knowledge of the form of the incoming attack. Generally, adversarial purifi…
MarkBERT: Marking Word Boundaries Improves Chinese BERT
Linyang Li, Yong Dai, Duyu Tang +3
We present a Chinese BERT model dubbed MarkBERT that uses word information in this work. Existing word-based BERT models regard words as basic units, however, due to the vocabulary…
Template-free Prompt Tuning for Few-shot NER
Ruotian Ma, Xin Zhou, Tao Gui +4
Prompt-based methods have been successfully applied in sentence-level few-shot learning tasks, mostly owing to the sophisticated design of templates and label words. However, when…