4 papers
A Layer-wise Analysis of Supervised Fine-Tuning
Qinghua Zhao, Xueling Gong, Xinyu Chen +2
While critical for alignment, Supervised Fine-Tuning (SFT) incurs the risk of catastrophic forgetting, yet the layer-wise emergence of instruction-following capabilities remains el…
EM-MIAs: Enhancing Membership Inference Attacks in Large Language Models through Ensemble Modeling
Zichen Song, Sitan Huang, Zhongfeng Kang
With the widespread application of large language models (LLM), concerns about the privacy leakage of model training data have increasingly become a focus. Membership Inference Att…
Layer Importance and Hallucination Analysis in Large Language Models via Enhanced Activation Variance-Sparsity
Zichen Song, Sitan Huang, Yuxin Wu +1
Evaluating the importance of different layers in large language models (LLMs) is crucial for optimizing model performance and interpretability. This paper first explores layer impo…
AVSS: Layer Importance Evaluation in Large Language Models via Activation Variance-Sparsity Analysis
Zichen Song, Yuxin Wu, Sitan Huang +1
The evaluation of layer importance in deep learning has been an active area of research, with significant implications for model optimization and interpretability. Recently, large…