67 citations · 221 across the 19 of their papers we have counts for
5 papers · 1 filter
Understanding the wiring evolution in differentiable neural architecture search
Sirui Xie, Shoukang Hu, Xinjiang Wang +4
Controversy exists on whether differentiable neural architecture search methods discover wiring topology effectively. To understand how wiring topology evolves, we study the underl…
AdaX: Adaptive Gradient Descent with Exponential Long Term Memory
Wenjie Li, Zhaoyang Zhang, Xinjiang Wang +1
Although adaptive optimization algorithms such as Adam show fast convergence in many machine learning tasks, this paper identifies a problem of Adam by analyzing its performance in…
Convolution-Weight-Distribution Assumption: Rethinking the Criteria of Channel Pruning
Zhongzhan Huang, Wenqi Shao, Xinjiang Wang +2
Channel pruning is a popular technique for compressing convolutional neural networks (CNNs), where various pruning criteria have been proposed to remove the redundant filters. From…
How Does BN Increase Collapsed Neural Network Filters?
Sheng Zhou, Xinjiang Wang, Ping Luo +3
Improving sparsity of deep neural networks (DNNs) is essential for network compression and has drawn much attention. In this work, we disclose a harmful sparsifying process called…
Towards Understanding Regularization in Batch Normalization
Ping Luo, Xinjiang Wang, Wenqi Shao +1
Batch Normalization (BN) improves both convergence and generalization in training neural networks. This work understands these phenomena theoretically. We analyze BN by using a bas…