3 citations · 13 across the 25 of their papers we have counts for
9 papers · 1 filter
Towards a Statistical Understanding of Neural Networks: Beyond the Neural Tangent Kernel Theories
Yicheng Li, Haobo Zhang, Jianfa Lai +2
A primary advantage of neural networks lies in their feature learning characteristics, which is challenging to theoretically analyze due to the complexity of their training dynamic…
Improving Adaptivity via Over-Parameterization in Sequence Models
Yicheng Li, Qian Lin
It is well known that eigenfunctions of a kernel play a crucial role in kernel regression. Through several examples, we demonstrate that even with the same set of eigenfunctions, t…
On the Impacts of the Random Initialization in the Neural Tangent Kernel Theory
Guhan Chen, Yicheng Li, Qian Lin
This paper aims to discuss the impact of random initialization of neural networks in the neural tangent kernel (NTK) theory, which is ignored by most recent works in the NTK theory…
On the Pinsker bound of inner product kernel regression in large dimensions
Weihao Lu, Jialin Ding, Haobo Zhang +1
Building on recent studies of large-dimensional kernel regression, particularly those involving inner product kernels on the sphere , we investigate the Pinsker bou…
On the Saturation Effect of Kernel Ridge Regression
Yicheng Li, Haobo Zhang, Qian Lin
The saturation effect refers to the phenomenon that the kernel ridge regression (KRR) fails to achieve the information theoretical lower bound when the smoothness of the undergroun…
The phase diagram of kernel interpolation in large dimensions
Haobo Zhang, Weihao Lu, Qian Lin
The generalization ability of kernel interpolation in large dimensions (i.e., for some ) might be one of the most interesting problems in the recent renaissance…
The Optimality of Kernel Classifiers in Sobolev Space
Jianfa Lai, Zhifan Li, Dongming Huang +1
Kernel methods are widely used in machine learning, especially for classification problems. However, the theoretical analysis of kernel classification is still limited. This paper…
Optimal Rates of Kernel Ridge Regression under Source Condition in Large Dimensions
Haobo Zhang, Yicheng Li, Weihao Lu +1
Motivated by the studies of neural networks (e.g.,the neural tangent kernel theory), we perform a study on the large-dimensional behavior of kernel ridge regression (KRR) where the…
Generalization Error Curves for Analytic Spectral Algorithms under Power-law Decay
Yicheng Li, Weiye Gan, Zuoqiang Shi +1
The generalization error curve of certain kernel regression method aims at determining the exact order of generalization error with various source condition, noise level and choice…