259 citations · 1.2k across the 48 of their papers we have counts for
6 papers · 1 filter
Trust Region Based Adversarial Attack on Neural Networks
Zhewei Yao, Amir Gholami, Peng Xu +2
Deep Neural Networks are quite vulnerable to adversarial perturbations. Current state-of-the-art adversarial attack methods typically require very time consuming hyper-parameter tu…
Parameter Re-Initialization through Cyclical Batch Size Schedules
Norman Mu, Zhewei Yao, Amir Gholami +2
Optimal parameter initialization remains a crucial problem for neural network training. A poor weight initialization may take longer to train and/or converge to sub-optimal solutio…
On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent
Noah Golmant, Nikita Vemuri, Zhewei Yao +5
Increasing the mini-batch size for stochastic gradient descent offers significant opportunities to reduce wall-clock training time, but there are a variety of theoretical and syste…
Large batch size training of neural networks with adversarial training and second-order information
Zhewei Yao, Amir Gholami, Daiyaan Arfeen +4
The most straightforward method to accelerate Stochastic Gradient Descent (SGD) computation is to distribute the randomly selected batch of inputs over multiple processors. To keep…
Inexact Non-Convex Newton-Type Methods
Zhewei Yao, Peng Xu, Farbod Roosta-Khorasani +1
For solving large-scale non-convex problems, we propose inexact variants of trust region and adaptive cubic regularization methods, which, to increase efficiency, incorporate vario…
Hessian-based Analysis of Large Batch Training and Robustness to Adversaries
Zhewei Yao, Amir Gholami, Qi Lei +2
Large batch size training of Neural Networks has been shown to incur accuracy loss when trained with the current methods. The exact underlying reasons for this are still not comple…