3 citations · 7 across the 11 of their papers we have counts for
12 papers
AgenticQwen: Training Small Agentic Language Models with Dual Data Flywheels for Industrial-Scale Tool Use
Yuanjie Lyu, Chengyu Wang, Haonan Zheng +4
Modern industrial applications increasingly demand language models that act as agents, capable of multi-step reasoning and tool use in real-world settings. These tasks are typicall…
Thinking with DistilQwen: A Tale of Four Distilled Reasoning and Reward Model Series
Wenrui Cai, Chengyu Wang, Junbing Yan +2
Recently, the demand for small and efficient reasoning models to support real-world applications has driven the development of knowledge distillation techniques that balance reason…
EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models
Chengyu Wang, Junbing Yan, Wenrui Cai +2
In this paper, we present EasyDistill, a comprehensive toolkit designed for effective black-box and white-box knowledge distillation (KD) of large language models (LLMs). Our frame…
Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations
Wenrui Cai, Chengyu Wang, Junbing Yan +2
The emergence of large reasoning models (LRMs) has transformed Natural Language Processing by excelling in complex tasks such as mathematical problem-solving and code generation. T…
DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models
Chengyu Wang, Junbing Yan, Yuanhao Yue +1
Enhancing computational efficiency and reducing deployment costs for large language models (LLMs) have become critical challenges in various resource-constrained scenarios. In this…
Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignment
Wenrui Cai, Chengyu Wang, Junbing Yan +2
The reasoning capabilities of large reasoning models (LRMs), such as OpenAI's o1 and DeepSeek-R1, have seen substantial advancements through deep thinking. However, these enhanceme…
On the Role of Long-tail Knowledge in Retrieval Augmented Large Language Models
Dongyang Li, Junbing Yan, Taolin Zhang +5
Retrieval augmented generation (RAG) exhibits outstanding performance in promoting the knowledge capabilities of large language models (LLMs) with retrieved documents related to us…
TRELM: Towards Robust and Efficient Pre-training for Knowledge-Enhanced Language Models
Junbing Yan, Chengyu Wang, Taolin Zhang +5
KEPLMs are pre-trained models that utilize external knowledge to enhance language understanding. Previous language models facilitated knowledge acquisition by incorporating knowled…
Do Large Language Models Understand Logic or Just Mimick Context?
Junbing Yan, Chengyu Wang, Jun Huang +1
Over the past few years, the abilities of large language models (LLMs) have received extensive attention, which have performed exceptionally well in complicated scenarios such as l…
Towards Better Parameter-Efficient Fine-Tuning for Large Language Models: A Position Paper
Chengyu Wang, Junbing Yan, Wei Zhang +1
This paper delves into the pressing need in Parameter-Efficient Fine-Tuning (PEFT) for Large Language Models (LLMs). While LLMs possess remarkable capabilities, their extensive par…
From Complex to Simple: Unraveling the Cognitive Tree for Reasoning with Small Language Models
Junbing Yan, Chengyu Wang, Taolin Zhang +3
Reasoning is a distinctive human capacity, enabling us to address complex problems by breaking them down into a series of manageable cognitive steps. Yet, complex logical reasoning…
Making Small Language Models Better Multi-task Learners with Mixture-of-Task-Adapters
Yukang Xie, Chengyu Wang, Junbing Yan +3
Recently, Large Language Models (LLMs) have achieved amazing zero-shot learning performance over a variety of Natural Language Processing (NLP) tasks, especially for text generativ…