Does equivariance matter at scale?

Johann Brehmer, Sönke Behrends, Pim de Haan +1

Given large datasets and sufficient compute, is it beneficial to design neural architectures for the structure and symmetries of each problem? Or is it more efficient to learn them…