753 citations · 974 across the 2 of their papers we have counts for
2 papers
cs.CV2019★ 221 cited
Stand-Alone Self-Attention in Vision Models
Prajit Ramachandran, Niki Parmar, Ashish Vaswani +3
Convolutions are a fundamental building block of modern computer vision systems. Recent approaches have argued for going beyond convolutions in order to capture long-range dependen…
cs.NE2017★ 753 cited
Searching for Activation Functions
Prajit Ramachandran, Barret Zoph, Quoc V. Le
The choice of activation functions in deep networks has a significant effect on the training dynamics and task performance. Currently, the most successful and widely-used activatio…