221 citations · 746 across the 6 of their papers we have counts for
6 papers · 1 filter
Revisiting 3D ResNets for Video Recognition
Xianzhi Du, Yeqing Li, Yin Cui +3
A recent work from Bello shows that training and scaling strategies may be more significant than model architectures for visual recognition. This short note studies effective train…
Revisiting ResNets: Improved Training and Scaling Strategies
Irwan Bello, William Fedus, Xianzhi Du +5
Novel computer vision architectures monopolize the spotlight, but the impact of the model architecture is often conflated with simultaneous changes to training methodology and scal…
LambdaNetworks: Modeling Long-Range Interactions Without Attention
Irwan Bello
We present lambda layers -- an alternative framework to self-attention -- for capturing long-range interactions between an input and structured contextual information (e.g. a pixel…
Global Self-Attention Networks for Image Recognition
Zhuoran Shen, Irwan Bello, Raviteja Vemulapalli +2
Recently, a series of works in computer vision have shown promising results on various image and video understanding tasks using self-attention. However, due to the quadratic compu…
Stand-Alone Self-Attention in Vision Models
Prajit Ramachandran, Niki Parmar, Ashish Vaswani +3
Convolutions are a fundamental building block of modern computer vision systems. Recent approaches have argued for going beyond convolutions in order to capture long-range dependen…
Attention Augmented Convolutional Networks
Irwan Bello, Barret Zoph, Ashish Vaswani +2
Convolutional networks have been the paradigm of choice in many computer vision applications. The convolution operation however has a significant weakness in that it only operates…