Speed Limit for Information Acquisition in Stochastic Learning Dynamics

Shuta Kobayashi, Andreas Dechant

Neural networks acquire internal representations through learning. In this work, we formulate stochastic gradient descent (SGD) as a Markovian stochastic process and derive a Fishe…