6 papers
TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning
Jinyang Wu, Chonghua Liao, Mingkuan Feng +6
Reinforcement learning (RL) has emerged as an effective paradigm for enhancing model reasoning. However, existing RL methods like GRPO typically rely on unstructured self-sampling…
Two-Stage Regularization-Based Structured Pruning for LLMs
Mingkuan Feng, Jinyang Wu, Siyuan Liu +7
The deployment of large language models (LLMs) is largely hindered by their large number of parameters. Structural pruning has emerged as a promising solution. Prior structured pru…
Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
Ruihan Jin, Pengpeng Shao, Zhengqi Wen +5
Knowledge distillation has emerged as a pivotal technique for transferring knowledge from stronger large language models (LLMs) to smaller, more efficient models. However, traditio…
RadialRouter: Structured Representation for Efficient and Robust Large Language Models Routing
Ruihan Jin, Pengpeng Shao, Zhengqi Wen +4
The rapid advancements in large language models (LLMs) have led to the emergence of routing techniques, which aim to efficiently select the optimal LLM from diverse candidates to t…
DReSS: Data-driven Regularized Structured Streamlining for Large Language Models
Mingkuan Feng, Jinyang Wu, Shuai Zhang +5
Large language models (LLMs) have achieved significant progress across various domains, but their increasing scale results in high computational and memory costs. Recent studies ha…
Pandora's Box or Aladdin's Lamp: A Comprehensive Analysis Revealing the Role of RAG Noise in Large Language Models
Jinyang Wu, Shuai Zhang, Feihu Che +4
Retrieval-Augmented Generation (RAG) has emerged as a crucial method for addressing hallucinations in large language models (LLMs). While recent research has extended RAG models to…