4 papers
How Long Does Infinite Width Last? Signal Propagation in Long-Range Linear Recurrences
Mariia Seleznova
We study signal propagation in linear recurrent models at finite width. While existing signal propagation theory relies predominantly on the infinite-width limit, it remains unclea…
GradPCA: Leveraging NTK Alignment for Reliable Out-of-Distribution Detection
Mariia Seleznova, Hung-Hsu Chou, Claudio Mayrink Verdun +1
We introduce GradPCA, an Out-of-Distribution (OOD) detection method that exploits the low-rank structure of neural network gradients induced by Neural Tangent Kernel (NTK) alignmen…
Latent Structure Emergence in Diffusion Models via Confidence-Based Filtering
Wei Wei, Yizhou Zeng, Kuntian Chen +3
Diffusion models rely on a high-dimensional latent space of initial noise seeds, yet it remains unclear whether this space contains sufficient structure to predict properties of th…
Revisiting Glorot Initialization for Long-Range Linear Recurrences
Noga Bar, Mariia Seleznova, Yotam Alexander +2
Proper initialization is critical for Recurrent Neural Networks (RNNs), particularly in long-range reasoning tasks, where repeated application of the same weight matrix can cause v…