89 citations · 133 across the 14 of their papers we have counts for
Showing 2022Show all
2 papers · 1 filter
cs.LG2022★ 89 cited
Transformers learn in-context by gradient descent
Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo +4
At present, the mechanisms of in-context learning in Transformers are not well understood and remain mostly an intuition. In this paper, we suggest that training Transformers on au…
cs.LG2022★ 1 cited
Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel
Seijin Kobayashi, Pau Vilimelis Aceituno, Johannes von Oswald
Identifying unfamiliar inputs, also known as out-of-distribution (OOD) detection, is a crucial property of any decision making process. A simple and empirically validated technique…