1.2k citations · 2.5k across the 23 of their papers we have counts for
4 papers · 1 filter
DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimization
Lin Xiao, Adams Wei Yu, Qihang Lin +1
Machine learning with big data often involves large optimization models. For distributed optimization over a cluster of machines, frequent communication and synchronization of all…
Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks
Lei Huang, Xianglong Liu, Bo Lang +3
Orthogonal matrix has shown advantages in training Recurrent Neural Networks (RNNs), but such matrix is limited to be square for the hidden-to-hidden transformation in RNNs. In thi…
Block-Normalized Gradient Method: An Empirical Study for Training Deep Neural Network
Adams Wei Yu, Lei Huang, Qihang Lin +2
In this paper, we propose a generic and simple strategy for utilizing stochastic gradient information in optimization. The technique essentially contains two consecutive steps in e…
Learning to Skim Text
Adams Wei Yu, Hongrae Lee, Quoc V. Le
Recurrent Neural Networks are showing much promise in many sub-areas of natural language processing, ranging from document classification to machine translation to automatic questi…