39 citations · 124 across the 17 of their papers we have counts for
18 papers
"Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach
Lingyu Gu, Yongqi Du, Yuan Zhang +4
Modern deep neural networks (DNNs) are extremely powerful; however, this comes at the price of increased depth and having more parameters per layer, making their training and infer…
On the Equivalence between Implicit and Explicit Neural Networks: A High-dimensional Viewpoint
Zenan Ling, Zhenyu Liao, Robert C. Qiu
Implicit neural networks have demonstrated remarkable success in various tasks. However, there is a lack of theoretical analysis of the connections and differences between implicit…
Robustness of random-control quantum-state tomography
Jingcheng Wang, Shaoliang Zhang, Jianming Cai +3
In a recently demonstrated quantum-state tomography scheme [Phys. Rev. Lett. 124, 010405 (2020)], a random control field is locally applied to a multipartite system to reconstruct…
An Achievable and Analytic Solution to Information Bottleneck for Gaussian Mixtures
Yi Song, Kai Wan, Zhenyu Liao +1
In this paper, we study a remote source coding scenario in which binary phase shift keying (BPSK) modulation sources are corrupted by additive white Gaussian noise (AWGN). An inter…
Random matrices in service of ML footprint: ternary random features with no performance loss
Hafiz Tiomoko Ali, Zhenyu Liao, Romain Couillet
In this article, we investigate the spectral behavior of random features kernel matrices of the type ${\bf K} = \mathbb{E}_{\bf w} \left[σ\left({\bf w}^{\sf T}{\bf x}_i\right)σ\lef…
Hessian Eigenspectra of More Realistic Nonlinear Models
Zhenyu Liao, Michael W. Mahoney
Given an optimization problem, the Hessian matrix and its eigenspectrum can be used in many ways, ranging from designing more efficient second-order algorithms to performing model…
Sparse sketches with small inversion bias
Michał Dereziński, Zhenyu Liao, Edgar Dobriban +1
For a tall matrix and a random sketching matrix , the sketched estimate of the inverse covariance matrix is typically biased: $E[(\…
Sparse Quantized Spectral Clustering
Zhenyu Liao, Romain Couillet, Michael W. Mahoney
Given a large data matrix, sparsifying, quantizing, and/or performing other entry-wise nonlinear operations can have numerous benefits, ranging from speeding up iterative algorithm…
Kernel regression in high dimensions: Refined analysis beyond double descent
Fanghui Liu, Zhenyu Liao, Johan A. K. Suykens
In this paper, we provide a precise characterization of generalization properties of high dimensional kernel ridge regression across the under- and over-parameterized regimes, depe…
A Random Matrix Analysis of Random Fourier Features: Beyond the Gaussian Kernel, a Precise Phase Transition, and the Corresponding Double Descent
Zhenyu Liao, Romain Couillet, Michael W. Mahoney
This article characterizes the exact asymptotics of random Fourier feature (RFF) regression, in the realistic setting where the number of data samples , their dimension , and…
Inner-product Kernels are Asymptotically Equivalent to Binary Discrete Kernels
Zhenyu Liao, Romain Couillet
This article investigates the eigenspectrum of the inner product-type kernel matrix under a bina…
High Dimensional Classification via Regularized and Unregularized Empirical Risk Minimization: Precise Error and Optimal Loss
Xiaoyi Mai, Zhenyu Liao
This article provides, through theoretical analysis, an in-depth understanding of the classification performance of the empirical risk minimization framework, in both ridge-regular…
A Geometric Approach of Gradient Descent Algorithms in Linear Neural Networks
Yacine Chitour, Zhenyu Liao, Romain Couillet
In this paper, we propose a geometric framework to analyze the convergence properties of gradient descent trajectories in the context of linear neural networks. We translate a well…
The Dynamics of Learning: A Random Matrix Approach
Zhenyu Liao, Romain Couillet
Understanding the learning dynamics of neural networks is one of the key issues for the improvement of optimization algorithms as well as for the theoretical comprehension of why d…
On the Spectrum of Random Features Maps of High Dimensional Data
Zhenyu Liao, Romain Couillet
Random feature maps are ubiquitous in modern statistical machine learning, where they generalize random projections by means of powerful, yet often difficult to analyze nonlinear o…
A Random Matrix Approach to Neural Networks
Cosme Louart, Zhenyu Liao, Romain Couillet
This article studies the Gram random matrix model , , classically found in the analysis of random feature maps and random neural networks, where $X=[x…
A Large Dimensional Analysis of Least Squares Support Vector Machines
Zhenyu Liao, Romain Couillet
In this article, a large dimensional performance analysis of kernel least squares support vector machines (LS-SVMs) is provided under the assumption of a two-class Gaussian mixture…
Random matrices meet machine learning: a large dimensional analysis of LS-SVM
Zhenyu Liao, Romain Couillet
This article proposes a performance analysis of kernel least squares support vector machines (LS-SVMs) based on a random matrix approach, in the regime where both the dimension of…