2 papers
cs.LG2021
Weighted Neural Tangent Kernel: A Generalized and Improved Network-Induced Kernel
Lei Tan, Shutong Wu, Xiaolin Huang
The Neural Tangent Kernel (NTK) has recently attracted intense study, as it describes the evolution of an over-parameterized Neural Network (NN) trained by gradient descent. Howeve…
cs.LG2021
Low Dimensional Landscape Hypothesis is True: DNNs can be Trained in Tiny Subspaces
Tao Li, Lei Tan, Qinghua Tao +2
Deep neural networks (DNNs) usually contain massive parameters, but there is redundancy such that it is guessed that the DNNs could be trained in low-dimensional subspaces. In this…