790 citations · 2k across the 26 of their papers we have counts for
Showing 2017Show all
2 papers · 1 filter
cs.CL2017
Improving Language Modeling using Densely Connected Recurrent Neural Networks
Fréderic Godin, Joni Dambre, Wesley De Neve
In this paper, we introduce the novel concept of densely connected layers into recurrent neural networks. We evaluate our proposed architecture on the Penn Treebank language modeli…
cs.CL2017★ 47 cited
Dual Rectified Linear Units (DReLUs): A Replacement for Tanh Activation Functions in Quasi-Recurrent Neural Networks
Fréderic Godin, Jonas Degrave, Joni Dambre +1
In this paper, we introduce a novel type of Rectified Linear Unit (ReLU), called a Dual Rectified Linear Unit (DReLU). A DReLU, which comes with an unbounded positive and negative…