2 citations · 4 across the 8 of their papers we have counts for
Showing 2020Show all
2 papers · 1 filter
cs.NE2020★ 1 cited
Regularized Flexible Activation Function Combinations for Deep Neural Networks
Renlong Jie, Junbin Gao, Andrey Vasnev +1
Activation in deep neural networks is fundamental to achieving non-linear mappings. Traditional studies mainly focus on finding fixed activations for a particular set of learning t…
cs.LG2020
Adaptive Hierarchical Hyper-gradient Descent
Renlong Jie, Junbin Gao, Andrey Vasnev +1
In this study, we investigate learning rate adaption at different levels based on the hyper-gradient descent framework and propose a method that adaptively learns the optimizer par…