11 papers
Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors
Naïl B. Khelifa, Richard E. Turner, Ramji Venkataramanan
Score-based diffusion models are typically trained by minimizing the score matching error, and standard theoretical analyses rely on this quantity to bound the sampling discr…
Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization
Naïl B. Khelifa, Richard E. Turner, Ramji Venkataramanan
Recursive training of generative models on their own outputs can lead to model collapse, a compounding drift away from the true data distribution. Existing theoretical works bound…
Quantifying Error Propagation and Model Collapse in Diffusion Models
Nail B. Khelifa, Richard E. Turner, Ramji Venkataramanan
Machine learning models are increasingly trained or fine-tuned on synthetic data. Recursively training on such data has been observed to significantly degrade performance in a wide…
Inferring Change Points in Regression via Sample Weighting
Gabriel Arpino, Ramji Venkataramanan
We study the problem of identifying change points in high-dimensional generalized linear models, and propose an approach based on sample-weighted empirical risk minimization. Our m…
Optimal Estimation in Orthogonally Invariant Generalized Linear Models: Spectral Initialization and Approximate Message Passing
Yihan Zhang, Hong Chang Ji, Ramji Venkataramanan +1
We consider the problem of parameter estimation from a generalized linear model with a random design matrix that is orthogonally invariant in law. Such a model allows the design ha…
Precise Asymptotics for Spectral Methods in Mixed Generalized Linear Models
Yihan Zhang, Marco Mondelli, Ramji Venkataramanan
In a mixed generalized linear model, the goal is to learn multiple signals from unlabeled observations: each sample comes from exactly one signal, but it is not known which one. We…