297 citations · 366 across the 9 of their papers we have counts for
3 papers · 1 filter
Efficient Image Generation with Variadic Attention Heads
Steven Walton, Ali Hassani, Xingqian Xu +2
While the integration of transformers in vision models have yielded significant improvements on vision tasks they still require significant amounts of computation for both training…
Design Amortization for Bayesian Optimal Experimental Design
Noble Kennamer, Steven Walton, Alexander Ihler
Bayesian optimal experimental design is a sub-field of statistics focused on developing methods to make efficient use of experimental resources. Any potential design is evaluated i…
Neighborhood Attention Transformer
Ali Hassani, Steven Walton, Jiachen Li +2
We present Neighborhood Attention (NA), the first efficient and scalable sliding-window attention mechanism for vision. NA is a pixel-wise operation, localizing self attention (SA)…