226 citations · 1.9k across the 128 of their papers we have counts for
50 papers · 1 filter
Percolation Dynamics in Optimization: Variance Cascades and Nested Symmetry
Sai Niranjan Ramachandran, Suvrit Sra
We study the dynamics of Stochastic Gradient Descent (SGD), which is known to steer deep neural networks toward invariant sets that correspond to simpler subnetworks. How this stee…
Trees to Flows and Back: Unifying Decision Trees and Diffusion Models
Sai Niranjan Ramachandran, Suvrit Sra
Decision trees and diffusion models are ostensibly disparate model classes, one discrete and hierarchical, the other continuous and dynamic. This work unifies the two by establishi…
Cost-Driven Representation Learning for Linear Quadratic Gaussian Control: Part II
Yi Tian, Kaiqing Zhang, Russ Tedrake +1
We study the problem of state representation learning for control from partial and potentially high-dimensional observations. We approach this problem via cost-driven state represe…
Cross-fluctuation phase transitions reveal sampling dynamics in diffusion models
Sai Niranjan Ramachandran, Manish Krishan Lal, Suvrit Sra
We analyse how the sampling dynamics of distributions evolve in score-based diffusion models using cross-fluctuations, a centered-moment statistic from statistical physics. Specifi…
A projection-based framework for gradient-free and parallel learning
Andreas Bergmeister, Manish Krishan Lal, Stefanie Jegelka +1
We present a feasibility-seeking approach to neural network training. This mathematical optimization framework is distinct from conventional gradient-based loss minimization and us…
Implicit Bias in Matrix Factorization and its Explicit Realization in a New Architecture
Yikun Hou, Suvrit Sra, Alp Yurtsever
Gradient descent for matrix factorization exhibits an implicit bias toward approximately low-rank solutions. While existing theories often assume the boundedness of iterates, empir…