When Representations Align: Universality in Representation Learning Dynamics

Loek van Rossem, Andrew M. Saxe

Deep neural networks come in many sizes and architectures. The choice of architecture, in conjunction with the dataset and learning algorithm, is commonly understood to affect the…