48 citations · 67 across the 3 of their papers we have counts for
3 papers · 1 filter
Inductive Biases and Variable Creation in Self-Attention Mechanisms
Benjamin L. Edelman, Surbhi Goel, Sham Kakade +1
Self-attention, an architectural motif designed to model long-range interactions in sequential data, has driven numerous recent breakthroughs in natural language processing and bey…
Causal Strategic Linear Regression
Yonadav Shavit, Benjamin Edelman, Brian Axelrod
In many predictive decision-making scenarios, such as credit scoring and academic testing, a decision-maker must construct a model that accounts for agents' propensity to "game" th…
SGD on Neural Networks Learns Functions of Increasing Complexity
Preetum Nakkiran, Gal Kaplun, Dimitris Kalimeris +4
We perform an experimental study of the dynamics of Stochastic Gradient Descent (SGD) in learning deep neural networks for several real and synthetic classification tasks. We show…