8 citations · 9 across the 2 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2022★ 8 cited
Position Prediction as an Effective Pretraining Strategy
Shuangfei Zhai, Navdeep Jaitly, Jason Ramapuram +7
Transformers have gained increasing popularity in a wide range of applications, including Natural Language Processing (NLP), Computer Vision and Speech Recognition, because of thei…
cs.LG2022★ 1 cited
Efficient Representation Learning via Adaptive Context Pooling
Chen Huang, Walter Talbott, Navdeep Jaitly +1
Self-attention mechanisms model long-range context by using pairwise attention between all input tokens. In doing so, they assume a fixed attention granularity defined by the indiv…