The Flip Side of the Reweighted Coin: Duality of Adaptive Dropout and Regularization

Daniel LeJeune, Hamid Javadi, Richard G. Baraniuk

Among the most successful methods for sparsifying deep (neural) networks are those that adaptively mask the network weights throughout training. By examining this masking, or dropo…