Recurrent Dropout without Memory Loss

Stanislau Semeniuta, Aliaksei Severyn, Erhardt Barth

This paper presents a novel approach to recurrent neural network (RNN) regularization. Differently from the widely adopted dropout method, which is applied to \textit{forward} conn…