107 citations · 271 across the 12 of their papers we have counts for
12 papers
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage
Yu Gui, Cong Ma, Yiqiao Zhong
We investigate the role of projection heads, also known as projectors, within the encoder-projector framework (e.g., SimCLR) used in contrastive learning. We aim to demystify the o…
Tractability from overparametrization: The example of the negative perceptron
Andrea Montanari, Yiqiao Zhong, Kangjie Zhou
In the negative perceptron problem we are given data points , where is a -dimensional vector and is a binary l…
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training
Andrea Montanari, Yiqiao Zhong
Modern neural networks are often operated in a strongly overparametrized regime: they comprise so many parameters that they can interpolate the training set, even if actual labels…
A Selective Overview of Deep Learning
Jianqing Fan, Cong Ma, Yiqiao Zhong
Deep learning has arguably achieved tremendous success in recent years. In simple words, deep learning uses the composition of many nonlinear functions to model the complex depende…
Robust high dimensional factor models with applications to statistical machine learning
Jianqing Fan, Kaizheng Wang, Yiqiao Zhong +1
Factor models are a class of powerful statistical models that have been widely used to deal with dependent measurements that arise frequently from various applications from genomic…
Optimal Subspace Estimation Using Overidentifying Vectors via Generalized Method of Moments
Jianqing Fan, Yiqiao Zhong
Many statistical models seek relationship between variables via subspaces of reduced dimensions. For instance, in factor models, variables are roughly distributed around a low dime…
Entrywise Eigenvector Analysis of Random Matrices with Low Expected Rank
Emmanuel Abbe, Jianqing Fan, Kaizheng Wang +1
Recovering low-rank structures via eigenvector perturbation analysis is a common problem in statistical machine learning, such as in factor analysis, community detection, ranking,…
Near-optimal bounds for phase synchronization
Yiqiao Zhong, Nicolas Boumal
The problem of phase synchronization is to estimate the phases (angles) of a complex unit-modulus vector from their noisy pairwise relative measurements , where…
Eigenvector Under Random Perturbation: A Nonasymptotic Rayleigh-Schrödinger Theory
Yiqiao Zhong
Rayleigh-Schrödinger perturbation theory is a well-known theory in quantum mechanics and it offers useful characterization of eigenvectors of a perturbed matrix. Suppose and pe…
An Eigenvector Perturbation Bound and Its Application to Robust Covariance Estimation
Jianqing Fan, Weichen Wang, Yiqiao Zhong
In statistics and machine learning, people are often interested in the eigenvectors (or singular vectors) of certain matrices (e.g. covariance matrices, data matrices, etc). Howeve…
Robust Covariance Estimation for Approximate Factor Models
Jianqing Fan, Weichen Wang, Yiqiao Zhong
In this paper, we study robust covariance estimation under the approximate factor model with observed factors. We propose a novel framework to first estimate the initial joint cova…
Differentially Private Data Releasing for Smooth Queries with Synthetic Database Output
Chi Jin, Ziteng Wang, Junliang Huang +2
We consider accurately answering smooth queries while preserving differential privacy. A query is said to be -smooth if it is specified by a function defined on whose…