1 citations · 1 across the 7 of their papers we have counts for
3 papers · 1 filter
How are linear representations learned? Exact solutions to the dynamics of abstraction
William W. Yang, Andrew M. Saxe, Peter E. Latham
In artificial and biological neural networks, concepts are often encoded as consistent linear directions in representation space. In deep learning, this idea is known as the linear…
Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networks
Yedi Zhang, Peter E. Latham, Leena Chennuru Vankadara +1
In this short note we consider the gradient descent dynamics of deep scalar linear networks, , which enjoy exact time-course solutions for any integer d…
Optimal Learning Rate Schedule for Balancing Effort and Performance
Valentina Njaradi, Rodrigo Carrasco-Davis, Peter E. Latham +1
Learning how to learn efficiently is a fundamental challenge for biological agents and a growing concern for artificial ones. To learn effectively, an agent must regulate its learn…