61 citations · 297 across the 23 of their papers we have counts for
24 papers
Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Gaussian Matrix Product
Mufan Li, Jaume de Dios Pont, Mihai Nica +1
We study the squared singular value spectrum of a non-square product of independent real Gaussian matrices, equivalently the feature covariance spectrum of a deep linear neural net…
Using alternating de Bruijn sequences to construct de Bruijn tori
Matthew Kreitzer, Mihai Nica, Rajesh Pereira
A de Bruijn torus is the two dimensional generalization of a de Bruijn sequence. While some methods exist to generate these tori, only a few methods of construction are known. We p…
Network Degeneracy as an Indicator of Training Performance: Comparing Finite and Infinite Width Angle Predictions
Cameron Jakub, Mihai Nica
Neural networks are powerful functions with widespread use, but the theoretical behaviour of these functions is not fully understood. Creating deep neural networks by stacking many…
Dynamic Sparse Training with Structured Sparsity
Mike Lasby, Anna Golubeva, Utku Evci +2
Dynamic Sparse Training (DST) methods achieve state-of-the-art results in sparse neural network training, matching the generalization of dense models while enabling sparse training…
Depth Degeneracy in Neural Networks: Vanishing Angles in Fully Connected ReLU Networks on Initialization
Cameron Jakub, Mihai Nica
Despite remarkable performance on a variety of tasks, many properties of deep neural networks are not yet theoretically understood. One such mystery is the depth degeneracy phenome…
Bounding generalization error with input compression: An empirical study with infinite-width networks
Angus Galloway, Anna Golubeva, Mahmoud Salem +3
Estimating the Generalization Error (GE) of Deep Neural Networks (DNNs) is an important task that often relies on availability of held-out data. The ability to better predict GE ba…
The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization
Mufan Bill Li, Mihai Nica, Daniel M. Roy
The logit outputs of a feedforward neural network at initialization are conditionally Gaussian, given a random covariance matrix defined by the penultimate layer. In this work, we…
The Exponentially Tilted Gaussian Prior for Variational Autoencoders
Griffin Floto, Stefan Kremer, Mihai Nica
An important property for deep neural networks is the ability to perform robust out-of-distribution detection on previously unseen data. This property is essential for safety purpo…
Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review
Xinhai Zhang, Jianbo Tao, Kaige Tan +9
Scenario-based approaches have been receiving a huge amount of attention in research and engineering of automated driving systems. Due to the complexity and uncertainty of the driv…
RSK in last passage percolation: a unified approach
Duncan Dauvergne, Mihai Nica, Bálint Virág
We present a version of the RSK correspondence based on the Pitman transform and geometric considerations. This version unifies ordinary RSK, dual RSK and continuous RSK. We show t…
The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization
Mufan Bill Li, Mihai Nica, Daniel M. Roy
Theoretical results show that neural networks can be approximated by Gaussian processes in the infinite-width limit. However, for fully connected networks, it has been previously s…
Solving Elliptic Equations with Brownian Motion: Bias Reduction and Temporal Difference Learning
Cameron Martin, Hongyuan Zhang, Julia Costacurta +2
The Feynman-Kac formula provides a way to understand solutions to elliptic partial differential equations in terms of expectations of continuous time Markov processes. This connect…
One-sided reflected Brownian motions and the KPZ fixed point
Mihai Nica, Jeremy Quastel, Daniel Remenik
We consider the system of one-sided reflected Brownian motions which is in variational duality with Brownian last passage percolation. We show that it has integrable transition pro…
A Derivative-Free Method for Solving Elliptic Partial Differential Equations with Deep Neural Networks
Jihun Han, Mihai Nica, Adam R Stinchcombe
We introduce a deep neural network based method for solving a class of elliptic partial differential equations. We approximate the solution of the PDE with a deep neural network wh…
Finite Depth and Width Corrections to the Neural Tangent Kernel
Boris Hanin, Mihai Nica
We prove the precise scaling, at finite depth and width, for the mean and variance of the neural tangent kernel (NTK) in a randomly initialized ReLU network. The standard deviation…
Uniform convergence to the Airy line ensemble
Duncan Dauvergne, Mihai Nica, Bálint Virág
We show that classical integrable models of last passage percolation and the related nonintersecting random walks converge uniformly on compact sets to the Airy line ensemble. Our…
Solution of the Kolmogorov equation for TASEP
Mihai Nica, Jeremy Quastel, Daniel Remenik
We provide a direct and elementary proof that the formula obtained in [MQR17] for the TASEP transition probabilities for general (one-sided) initial data solves the Kolmogorov back…
Products of Many Large Random Matrices and Gradients in Deep Neural Networks
Boris Hanin, Mihai Nica
We study products of random matrices in the regime where the number of terms and the size of the matrices simultaneously tend to infinity. Our main theorem is that the logarithm of…
The landscape of the spiked tensor model
Gerard Ben Arous, Song Mei, Andrea Montanari +1
We consider the problem of estimating a large rank-one tensor , in Gaussian noise. Earlier work characterized…
Intermediate disorder limits for multi-layer semi-discrete directed polymers
Mihai Nica
We show that the partition function of the multi-layer semi-discrete directed polymer converges in the intermediate disorder regime to the partition function for the multi-layer co…
Intermediate disorder directed polymers and the multi-layer extension of the stochastic heat equation
Ivan Corwin, Mihai Nica
We consider directed polymer models involving multiple non-intersecting random walks moving through a space-time disordered environment in one spatial dimension. For a single rando…
Optimal Strategy in "Guess Who?": Beyond Binary Search
Mihai Nica
"Guess Who?" is a popular two player game where players ask "Yes"/"No" questions to search for their opponent's secret identity from a pool of possible candidates. This is modeled…
Decorated Young Tableaux and the Poissonized Robinson-Schensted Process
Mihai Nica
We introduce an object called a decorated Young tableau which can equivalently be viewed as a continuous time trajectory of Young diagrams or as a non-intersecting line ensemble. B…
Stabilization Time for a Type of Evolution on Binary Strings
Jacob Funk, Mihai Nica, Michael Noyes
We consider a type of evolution on {0,1}^n which occurs in discrete steps whereby at each step, we replace every occurrence of the substring "01" by "10". After at most n-1 steps w…