5 papers
Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective
Junpeng Zhang, Lei Cheng, Guoxi Zhang +3
This paper explores a scientific question in supervised fine-tuning (SFT): why SFT is broadly effective for small-scale deep neural networks, yet can produce inconsistent or even d…
Technical Report: Quantifying and Analyzing the Generalization Power of a DNN
Yuxuan He, Junpeng Zhang, Lei Cheng +2
This paper proposes a new perspective for analyzing the generalization power of deep neural networks (DNNs), i.e., directly disentangling and analyzing the dynamics of generalizabl…
Revisiting Generalization Power of a DNN in Terms of Symbolic Interactions
Lei Cheng, Junpeng Zhang, Qihan Ren +1
This paper aims to analyze the generalization power of deep neural networks (DNNs) from the perspective of interactions. Unlike previous analysis of a DNN's generalization power in…
Randomness of Low-Layer Parameters Determines Confusing Samples in Terms of Interaction Representations of a DNN
Junpeng Zhang, Lei Cheng, Qing Li +2
In this paper, we find that the complexity of interactions encoded by a deep neural network (DNN) can explain its generalization power. We also discover that the confusing samples…
Towards the Dynamics of a DNN Learning Symbolic Interactions
Qihan Ren, Junpeng Zhang, Yang Xu +3
This study proves the two-phase dynamics of a deep neural network (DNN) learning interactions. Despite the long disappointing view of the faithfulness of post-hoc explanation of a…