336 citations · 1.6k across the 53 of their papers we have counts for
6 papers · 1 filter
Covariant Gradient Descent
Dmitry Guskov, Vitaly Vanchurin
We present a manifestly covariant formulation of the gradient descent method, ensuring consistency across arbitrary coordinate systems and general curved trainable spaces. The opti…
Geometric Learning Dynamics
Vitaly Vanchurin
We present a unified geometric framework for modeling learning dynamics in physical, biological, and machine learning systems. The theory reveals three fundamental regimes, each em…
Dataset-learning duality and emergent criticality
Ekaterina Kukleva, Vitaly Vanchurin
In artificial neural networks, the activation dynamics of non-trainable variables is strongly coupled to the learning dynamics of trainable variables. During the activation pass, t…
Autonomous particles
Nikola Andrejic, Vitaly Vanchurin
Consider a reinforcement learning problem where an agent has access to a very large amount of information about the environment, but it can only take very few actions to accomplish…
Towards a theory of quantum gravity from neural networks
Vitaly Vanchurin
Neural network is a dynamical system described by two different types of degrees of freedom: fast-changing non-trainable variables (e.g. state of neurons) and slow-changing trainab…
Towards a theory of machine learning
Vitaly Vanchurin
We define a neural network as a septuple consisting of (1) a state vector, (2) an input projection, (3) an output projection, (4) a weight matrix, (5) a bias vector, (6) an activat…