Rethinking Efficacy of Softmax for Lightweight Non-Local Neural Networks

Yooshin Cho, Youngsoo Kim, Hanbyel Cho +3

Non-local (NL) block is a popular module that demonstrates the capability to model global contexts. However, NL block generally has heavy computation and memory costs, so it is imp…