37 citations · 63 across the 22 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.LG2022★ 4 cited
Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse
Lorenzo Noci, Sotiris Anagnostidis, Luca Biggio +3
Transformers have achieved remarkable success in several domains, ranging from natural language processing to computer vision. Nevertheless, it has been recently shown that stackin…
stat.ML2022★ 1 cited
Phenomenology of Double Descent in Finite-Width Neural Networks
Sidak Pal Singh, Aurelien Lucchi, Thomas Hofmann +1
`Double descent' delineates the generalization behaviour of models depending on the regime they belong to: under- or over-parameterized. The current theoretical understanding behin…