2 citations · 2 across the 6 of their papers we have counts for
7 papers
Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
Ruihan Jin, Pengpeng Shao, Zhengqi Wen +5
Knowledge distillation has emerged as a pivotal technique for transferring knowledge from stronger large language models (LLMs) to smaller, more efficient models. However, traditio…
RadialRouter: Structured Representation for Efficient and Robust Large Language Models Routing
Ruihan Jin, Pengpeng Shao, Zhengqi Wen +4
The rapid advancements in large language models (LLMs) have led to the emergence of routing techniques, which aim to efficiently select the optimal LLM from diverse candidates to t…
Two-Stage Regularization-Based Structured Pruning for LLMs
Mingkuan Feng, Jinyang Wu, Siyuan Liu +7
The deployment of large language models (LLMs) is largely hindered by their large number of parameters. Structural pruning has emerged as a promising solution. Prior structured pru…
TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning
Jinyang Wu, Chonghua Liao, Mingkuan Feng +6
Reinforcement learning (RL) has emerged as an effective paradigm for enhancing model reasoning. However, existing RL methods like GRPO typically rely on unstructured self-sampling…
DReSS: Data-driven Regularized Structured Streamlining for Large Language Models
Mingkuan Feng, Jinyang Wu, Shuai Zhang +5
Large language models (LLMs) have achieved significant progress across various domains, but their increasing scale results in high computational and memory costs. Recent studies ha…
Pandora's Box or Aladdin's Lamp: A Comprehensive Analysis Revealing the Role of RAG Noise in Large Language Models
Jinyang Wu, Shuai Zhang, Feihu Che +4
Retrieval-Augmented Generation (RAG) has emerged as a crucial method for addressing hallucinations in large language models (LLMs). While recent research has extended RAG models to…
Can large language models understand uncommon meanings of common words?
Jinyang Wu, Feihu Che, Xinxin Zheng +5
Large language models (LLMs) like ChatGPT have shown significant advancements across diverse natural language understanding (NLU) tasks, including intelligent dialogue and autonomo…