6 papers
An Equivariance Toolbox for Learning Dynamics
Yongyi Yang, Liu Ziyin
Many theoretical results in deep learning can be traced to symmetry or equivariance of neural networks under parameter transformations. However, existing analyses are typically pro…
When Reasoning Meets Its Laws
Junyu Zhang, Yifan Sun, Tianang Leng +4
Despite the superior performance of Large Reasoning Models (LRMs), their reasoning behaviors are often counterintuitive, leading to suboptimal reasoning capabilities. To theoretica…
Does Feedback Alignment Work at Biological Timescales?
Marc Gong Bacvanski, Liu Ziyin, Tomaso Poggio
Feedback alignment and related weight-transport-free algorithms are often proposed as biologically plausible alternatives to backpropagation, yet they are typically formulated in d…
A universal compression theory for lottery ticket hypothesis and neural scaling laws
Hong-Yi Wang, Di Luo, Tomaso Poggio +2
When training large-scale models, the performance typically scales with the number of parameters and the dataset size according to a slow power law. A fundamental theoretical and p…
Heterosynaptic Circuits Are Universal Gradient Machines
Liu Ziyin, Isaac Chuang, Tomaso Poggio
We propose a design principle for the learning circuits of the biological brain. The principle states that almost any dendritic weights updated via heterosynaptic plasticity can im…
Compositional Generalization via Forced Rendering of Disentangled Latents
Qiyao Liang, Daoyuan Qian, Liu Ziyin +1
Composition-the ability to generate myriad variations from finite means-is believed to underlie powerful generalization. However, compositional generalization remains a key challen…