6 papers
Algorithmic Separation between Constant-Depth and Logarithmic-Depth Neural Networks
Yunwei Ren, Zihao Wang, Jason D. Lee
Despite the empirical advantages of deep networks over shallow ones, theoretical depth separations largely concern approximation power, while algorithmic results are mostly limited…
Phase Transitions for Feature Learning in Neural Networks
Andrea Montanari, Zihao Wang
According to a popular viewpoint, neural networks learn from data by first identifying low-dimensional representations, and subsequently fitting the best model in this space. Recen…
Non-negative matrix factorization algorithms generally improve topic model fits
Peter Carbonetto, Abhishek Sarkar, Zihao Wang +1
In an effort to develop topic modeling methods that can be quickly applied to large data sets, we revisit the problem of maximum-likelihood estimation in topic models. It is known,…
Neural Networks Learn Generic Multi-Index Models Near Information-Theoretic Limit
Bohan Zhang, Zihao Wang, Hengyu Fu +1
In deep learning, a central issue is to understand how neural networks efficiently learn high-dimensional features. To this end, we explore the gradient descent learning of a gener…
A Linear Complexity Algorithm for Optimal Transport Problem with Log-type Cost
Ziyuan Lyu, Zihao Wang, Hao Wu +1
In [Q. Liao et al., Commun. Math. Sci., 20(2022)], a linear-time Sinkhorn algorithm is developed based on dynamic programming, which significantly reduces the computational complex…
Learning Hierarchical Polynomials of Multiple Nonlinear Features with Three-Layer Networks
Hengyu Fu, Zihao Wang, Eshaan Nichani +1
In deep learning theory, a critical question is to understand how neural networks learn hierarchical features. In this work, we study the learning of hierarchical polynomials of \t…