3 citations · 4 across the 6 of their papers we have counts for
6 papers
EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
Chengyu Wang, Junbing Yan, Wenrui Cai +2
In this paper, we present EasyDistill, a comprehensive toolkit designed for effective black-box and white-box knowledge distillation (KD) of large language models (LLMs). Our frame…
On the Role of Long-tail Knowledge in Retrieval Augmented Large Language Models
Dongyang Li, Junbing Yan, Taolin Zhang +5
Retrieval augmented generation (RAG) exhibits outstanding performance in promoting the knowledge capabilities of large language models (LLMs) with retrieved documents related to us…
TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models
Junbing Yan, Chengyu Wang, Taolin Zhang +5
KEPLMs are pre-trained models that utilize external knowledge to enhance language understanding. Previous language models facilitated knowledge acquisition by incorporating knowled…
Do Large Language Models Understand Logic or Just Mimick Context?
Junbing Yan, Chengyu Wang, Jun Huang +1
Over the past few years, the abilities of large language models (LLMs) have received extensive attention, which have performed exceptionally well in complicated scenarios such as l…
From Complex to Simple: Unraveling the Cognitive Tree for Reasoning with Small Language Models
Junbing Yan, Chengyu Wang, Taolin Zhang +3
Reasoning is a distinctive human capacity, enabling us to address complex problems by breaking them down into a series of manageable cognitive steps. Yet, complex logical reasoning…
Making Small Language Models Better Multi-task Learners with Mixture-of-Task-Adapters
Yukang Xie, Chengyu Wang, Junbing Yan +3
Recently, Large Language Models (LLMs) have achieved amazing zero-shot learning performance over a variety of Natural Language Processing (NLP) tasks, especially for text generativ…