7 papers
Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
Ruihan Jin, Pengpeng Shao, Zhengqi Wen +5
Knowledge distillation has emerged as a pivotal technique for transferring knowledge from stronger large language models (LLMs) to smaller, more efficient models. However, traditio…
Better, Stronger, Faster: Tackling the Trilemma in MLLM-based Segmentation with Simultaneous Textual Mask Prediction
Jiazhen Liu, Mingkuan Feng, Long Chen
Integrating segmentation into Multimodal Large Language Models (MLLMs) presents a core trilemma: simultaneously preserving dialogue ability, achieving high segmentation performance…
RadialRouter: Structured Representation for Efficient and Robust Large Language Models Routing
Ruihan Jin, Pengpeng Shao, Zhengqi Wen +4
The rapid advancements in large language models (LLMs) have led to the emergence of routing techniques, which aim to efficiently select the optimal LLM from diverse candidates to t…
Two-Stage Regularization-Based Structured Pruning for LLMs
Mingkuan Feng, Jinyang Wu, Siyuan Liu +7
The deployment of large language models (LLMs) is largely hindered by their large number of parameters. Structural pruning has emerged as a promising solution. Prior structured pru…
AStar: Boosting Multimodal Reasoning with Automated Structured Thinking
Jinyang Wu, Mingkuan Feng, Guocheng Zhai +7
Multimodal large language models excel across diverse domains but struggle with complex visual reasoning tasks. To enhance their reasoning capabilities, current approaches typicall…
DReSS: Data-driven Regularized Structured Streamlining for Large Language Models
Mingkuan Feng, Jinyang Wu, Shuai Zhang +5
Large language models (LLMs) have achieved significant progress across various domains, but their increasing scale results in high computational and memory costs. Recent studies ha…