Exploring Information-Theoretic Metrics Associated with Neural Collapse in Supervised Training

Kun Song, Zhiquan Tan, Bochao Zou +3

In this paper, we introduce matrix entropy as an analytical tool for studying supervised learning, investigating the information content of data representations and classification…