most citedShould Under-parameterized Student Networks Copy or Average Teacher Weights?

2 citations · 3 across the 3 of their papers we have counts for

collaborators
Showing cs.LGShow all

2 papers · 1 filter

cs.LG2024

Learning Gaussian Multi-Index Models with Gradient Flow: Time Complexity and Directional Convergence

Berfin Şimşek, Amire Bendjeddou, Daniel Hsu

This work focuses on the gradient flow dynamics of a neural network model that uses correlation loss to approximate a multi-index function on high-dimensional standard Gaussian dat…

cs.LG2023★ 2 cited

Should Under-parameterized Student Networks Copy or Average Teacher Weights?

Berfin Şimşek, Amire Bendjeddou, Wulfram Gerstner +1

Any continuous function can be approximated arbitrarily well by a neural network with sufficiently many neurons . We consider the case when itself is a neural networ…