Gated Orthogonal Recurrent Units: On Learning to Forget

Li Jing, Caglar Gulcehre, John Peurifoy +4

We present a novel recurrent neural network (RNN) based model that combines the remembering ability of unitary RNNs with the ability of gated RNNs to effectively forget redundant/i…