11 citations · 12 across the 10 of their papers we have counts for
Showing cs.CVShow all
3 papers · 1 filter
cs.CV2023★ 1 cited
Mimetic Initialization of Self-Attention Layers
Asher Trockman, J. Zico Kolter
It is notoriously difficult to train Transformers on small datasets; typically, large pre-trained models are instead used as the starting point. We explore the weights of such pre-…
cs.CV2022
Understanding the Covariance Structure of Convolutional Filters
Asher Trockman, Devin Willmott, J. Zico Kolter
Neural network weights are typically initialized at random from univariate distributions, controlling just the variance of individual weights even in highly-structured operations l…
cs.CV2022
Patches Are All You Need?
Asher Trockman, J. Zico Kolter
Although convolutional networks have been the dominant architecture for vision tasks for many years, recent experiments have shown that Transformer-based models, most notably the V…