8 citations · 23 across the 12 of their papers we have counts for
5 papers · 1 filter
What shapes the loss landscape of self-supervised learning?
Liu Ziyin, Ekdeep Singh Lubana, Masahito Ueda +1
Prevention of complete and dimensional collapse of representations has recently become a design principle for self-supervised learning (SSL). However, questions remain in our theor…
Exact Phase Transitions in Deep Learning
Liu Ziyin, Masahito Ueda
This work reports deep-learning-unique first-order and second-order phase transitions, whose phenomenology closely follows that in statistical physics. In particular, we prove that…
Universal Thermodynamic Uncertainty Relation in Non-Equilibrium Dynamics
Liu Ziyin, Masahito Ueda
We derive a universal thermodynamic uncertainty relation (TUR) that applies to an arbitrary observable in a general Markovian system. The generality of our result allows us to make…
Interplay between depth of neural networks and locality of target functions
Takashi Mori, Masahito Ueda
It has been recognized that heavily overparameterized deep neural networks (DNNs) exhibit surprisingly good generalization performance in various machine-learning tasks. Although b…
Stochastic Neural Networks with Infinite Width are Deterministic
Liu Ziyin, Hanlin Zhang, Xiangming Meng +3
This work theoretically studies stochastic neural networks, a main type of neural network in use. We prove that as the width of an optimized stochastic neural network tends to infi…