3 papers
stat.ML2026
The hidden advantage of mask resampling: a theory of masked autoencoders
Jorge Medina Moreira, Lorenzo Bardone, Lenka Zdeborová
Why can masked prediction learn useful representations that unmasked reconstruction misses? We study this question in a high-dimensional model of a masked autoencoder (MAE) trained…
cs.CL2026
Tracing Persona Vectors Through LLM Pretraining
Viktor Moskvoretskii, Dominik Glandorf, Jorge Medina Moreira +2
How large language models internally represent high-level behaviors is a core interpretability question with direct relevance to AI safety: it determines what we can detect, audit,…
stat.ML2026
A solvable high-dimensional model where nonlinear autoencoders learn structure invisible to PCA while test loss misaligns with generalization
Vicente Conde Mendes, Lorenzo Bardone, Cédric Koller +4
Many real-world datasets contain hidden structure that cannot be detected by simple linear correlations between input features. For example, latent factors may influence the data i…