R-Drop: Regularized Dropout for Neural Networks

Xiaobo Liang, Lijun Wu, Juntao Li +6

Dropout is a powerful and widely used technique to regularize the training of deep neural networks. In this paper, we introduce a simple regularization strategy upon dropout in mod…