86 citations · 118 across the 7 of their papers we have counts for
7 papers
A Unified Analysis of Stochastic Momentum Methods for Deep Learning
Yan Yan, Tianbao Yang, Zhe Li +2
Stochastic momentum methods have been widely adopted in training deep neural networks. However, their theoretical analysis of convergence of the training objective and the generali…
An Aggressive Genetic Programming Approach for Searching Neural Network Structure Under Computational Constraints
Zhe Li, Xuehan Xiong, Zhou Ren +3
Recently, there emerged revived interests of designing automatic programs (e.g., using genetic/evolutionary algorithms) to optimize the structure of Convolutional Neural Networks (…
EIGEN: Ecologically-Inspired GENetic Approach for Neural Network Structure Searching from Scratch
Jian Ren, Zhe Li, Jianchao Yang +3
Designing the structure of neural networks is considered one of the most challenging tasks in deep learning, especially when there is few prior knowledge about the task domain. In…
A Simple Analysis for Exp-concave Empirical Minimization with Arbitrary Convex Regularizer
Tianbao Yang, Zhe Li, Lijun Zhang
In this paper, we present a simple analysis of {\bf fast rates} with {\it high probability} of {\bf empirical minimization} for {\it stochastic composite optimization} over a finit…
SEP-Nets: Small and Effective Pattern Networks
Zhe Li, Xiaoyu Wang, Xutao Lv +1
While going deeper has been witnessed to improve the performance of convolutional neural networks (CNN), going smaller for CNN has received increasing attention recently due to its…
Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization
Tianbao Yang, Qihang Lin, Zhe Li
Recently, {\it stochastic momentum} methods have been widely adopted in training deep neural networks. However, their convergence analysis is still underexplored at the moment, in…
Improved Dropout for Shallow and Deep Learning
Zhe Li, Boqing Gong, Tianbao Yang
Dropout has been witnessed with great success in training deep neural networks by independently zeroing out the outputs of neurons at random. It has also received a surge of intere…