8 papers
CODEBLOCK: Learning to Supervise Code at the Right Granularity
Zhijie Deng, Ling Li, Jinlong Pang +4
Supervised fine-tuning of code LLMs typically applies uniform cross-entropy loss to all response tokens, implicitly assuming that every token provides equally useful learning signa…
Mapping Text to Multiplex Graph: Prompt Compression as Lévy Walk-Guided Graph Pruning
Yaxin Gao, Yao Lu, Jinhong Deng +7
Existing prompt compression methods treat text as flat token sequences, failing to capture the distributed nature of important information, which is often spread across multiple lo…
OFFSIDE: Benchmarking Unlearning Misinformation in Multimodal Large Language Models
Hao Zheng, Zirui Pang, Ling li +5
Advances in Multimodal Large Language Models (MLLMs) intensify concerns about data privacy, making Machine Unlearning (MU), the selective removal of learned information, a critical…
The Structural Scalpel: Automated Contiguous Layer Pruning for Large Language Models
Yao Lu, Yuqi Li, Wenbin Xie +4
Although large language models (LLMs) have achieved revolutionary breakthroughs in many fields, their large model size and high computational cost pose significant challenges for p…
LM-mixup: Text Data Augmentation via Language Model based Mixup
Zhijie Deng, Zhouan Shen, Ling Li +5
Instruction tuning is crucial for aligning Large Language Models (LLMs), yet the quality of instruction-following data varies significantly. While high-quality data is paramount, i…
SelectMix: Enhancing Label Noise Robustness through Targeted Sample Mixing
Qiuhao Liu, Ling Li, Yao Lu +3
Deep neural networks tend to memorize noisy labels, severely degrading their generalization performance. Although Mixup has demonstrated effectiveness in improving generalization a…