9 papers
Width-Independent Compressibility of Deep Neural Networks
Hong-Yi Wang, Mingze Wang, Liu Ziyin
It has long been known that well-trained neural networks can be compressed very strongly without affecting their performance, an important phenomenon that remains poorly understood…
An Equivariance Toolbox for Learning Dynamics
Yongyi Yang, Liu Ziyin
Many theoretical results in deep learning can be traced to symmetry or equivariance of neural networks under parameter transformations. However, existing analyses are typically pro…
When Reasoning Meets Its Laws
Junyu Zhang, Yifan Sun, Tianang Leng +4
Despite the superior performance of Large Reasoning Models (LRMs), their reasoning behaviors are often counterintuitive, leading to suboptimal reasoning capabilities. To theoretica…
Does Feedback Alignment Work at Biological Timescales?
Marc Gong Bacvanski, Liu Ziyin, Tomaso Poggio
Feedback alignment and related weight-transport-free algorithms are often proposed as biologically plausible alternatives to backpropagation, yet they are typically formulated in d…
A universal compression theory for lottery ticket hypothesis and neural scaling laws
Hong-Yi Wang, Di Luo, Tomaso Poggio +2
When training large-scale models, the performance typically scales with the number of parameters and the dataset size according to a slow power law. A fundamental theoretical and p…
Heterosynaptic Circuits Are Universal Gradient Machines
Liu Ziyin, Isaac Chuang, Tomaso Poggio
We propose a design principle for the learning circuits of the biological brain. The principle states that almost any dendritic weights updated via heterosynaptic plasticity can im…
Machine Learning Decoding of Circuit-Level Noise for Bivariate Bicycle Codes
John Blue, Harshil Avlani, Zhiyang He +2
Fault-tolerant quantum computers will depend crucially on the performance of the classical decoding algorithm which takes in the results of measurements and outputs corrections to…
Compositional Generalization via Forced Rendering of Disentangled Latents
Qiyao Liang, Daoyuan Qian, Liu Ziyin +1
Composition-the ability to generate myriad variations from finite means-is believed to underlie powerful generalization. However, compositional generalization remains a key challen…
Power Laws and Symmetries in a Minimal Model of Financial Market Economy
Liu Ziyin, Katsuya Ito, Kentaro Imajo +1
A financial market is a system resulting from the complex interaction between participants in a closed economy. We propose a minimal microscopic model of the financial market econo…