471 citations · 1.4k across the 25 of their papers we have counts for
3 papers · 1 filter
On the Quantitative Analysis of Decoder-Based Generative Models
Yuhuai Wu, Yuri Burda, Ruslan Salakhutdinov +1
The past several years have seen remarkable progress in generative models which produce convincing samples of images and other modalities. A shared component of many powerful gener…
On Multiplicative Integration with Recurrent Neural Networks
Yuhuai Wu, Saizheng Zhang, Ying Zhang +2
We introduce a general and simple structural design called Multiplicative Integration (MI) to improve recurrent neural networks (RNNs). MI changes the way in which information from…
Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations
Behnam Neyshabur, Yuhuai Wu, Ruslan Salakhutdinov +1
We investigate the parameter-space geometry of recurrent neural networks (RNNs), and develop an adaptation of path-SGD optimization method, attuned to this geometry, that can learn…