The distance between the weights of the neural network is meaningful
arXiv:2102.00396
Abstract
In the application of neural networks, we need to select a suitable model based on the problem complexity and the dataset scale. To analyze the network's capacity, quantifying the information learned by the network is necessary. This paper proves that the distance between the neural network weights in different training stages can be used to estimate the information accumulated by the network in the training process directly. The experiment results verify the utility of this method. An application of this method related to the label corruption is shown at the end.
arXiv comment10 pages, 13 figure
References in corpus (9)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- YOLOv3: An Incremental Improvement
- The information bottleneck method
- Shortcut Learning in Deep Neural Networks
- Understanding deep learning requires rethinking generalization
- Opening the Black Box of Deep Neural Networks via Information
- Deep Variational Information Bottleneck
- Where is the Information in a Deep Neural Network?
- Probing Neural Network Comprehension of Natural Language Arguments