9 papers
Breaking the Tie: A Cluster-Aware Routing Framework for Large Language Models
Yao Lu, Zhaiyuan Ji, Yaxin Gao +6
With the rapid development of artificial intelligence, the emergence of various Large Language Models (LLMs) has created a rich model ecosystem. However, this also brings a key cha…
Mapping Text to Multiplex Graph: Prompt Compression as Lévy Walk-Guided Graph Pruning
Yaxin Gao, Yao Lu, Jinhong Deng +7
Existing prompt compression methods treat text as flat token sequences, failing to capture the distributed nature of important information, which is often spread across multiple lo…
CODEBLOCK: Learning to Supervise Code at the Right Granularity
Zhijie Deng, Ling Li, Jinlong Pang +5
Supervised fine-tuning of code LLMs typically applies uniform cross-entropy loss to all response tokens, implicitly assuming that every token provides equally useful learning signa…
The Structural Scalpel: Automated Contiguous Layer Pruning for Large Language Models
Yao Lu, Yuqi Li, Wenbin Xie +4
Although large language models (LLMs) have achieved revolutionary breakthroughs in many fields, their large model size and high computational cost pose significant challenges for p…
OFFSIDE: Benchmarking Unlearning Misinformation in Multimodal Large Language Models
Hao Zheng, Zirui Pang, Ling li +5
Advances in Multimodal Large Language Models (MLLMs) intensify concerns about data privacy, making Machine Unlearning (MU), the selective removal of learned information, a critical…
LM-mixup: Text Data Augmentation via Language Model based Mixup
Zhijie Deng, Zhouan Shen, Ling Li +5
Instruction tuning is crucial for aligning Large Language Models (LLMs), yet the quality of instruction-following data varies significantly. While high-quality data is paramount, i…
SelectMix: Enhancing Label Noise Robustness through Targeted Sample Mixing
Qiuhao Liu, Ling Li, Yao Lu +3
Deep neural networks tend to memorize noisy labels, severely degrading their generalization performance. Although Mixup has demonstrated effectiveness in improving generalization a…
Better Reasoning with Less Data: Enhancing VLMs Through Unified Modality Scoring
Mingjie Xu, Andrew Estornell, Hongzheng Yang +4
The application of visual instruction tuning and other post-training techniques has significantly enhanced the capabilities of Large Language Models (LLMs) in visual understanding,…
GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detection
Zhijie Deng, Chris Yuhao Liu, Zirui Pang +5
Large Language Models (LLMs) have demonstrated strong capabilities in memorizing vast amounts of knowledge across diverse domains. However, the ability to selectively forget specif…