DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks

Lin Zehui, Pengfei Liu, Luyao Huang +3

Variants dropout methods have been designed for the fully-connected layer, convolutional layer and recurrent layer in neural networks, and shown to be effective to avoid overfittin…