Efficient batchwise dropout training using submatrices

Ben Graham, Jeremy Reizenstein, Leigh Robinson

Dropout is a popular technique for regularizing artificial neural networks. Dropout networks are generally trained by minibatch gradient descent with a dropout mask turning off som…