12 papers
Width-Independent Compressibility of Deep Neural Networks
Hong-Yi Wang, Mingze Wang, Liu Ziyin
It has long been known that well-trained neural networks can be compressed very strongly without affecting their performance, an important phenomenon that remains poorly understood…
RippleMem: From Isolated Retrieval to Associative Recollection for Long-Term Agent Memory
Jingbo Ji, Lingyi Li, Xilong Cheng +4
LLM-based agents increasingly rely on external memory to support long-horizon reasoning and interaction. However, the main bottleneck is not simply storing past experience, but rec…
Machine Learning Decoding of Circuit-Level Noise for Bivariate Bicycle Codes
John Blue, Harshil Avlani, Zhiyang He +2
Fault-tolerant quantum computers will depend crucially on the performance of the classical decoding algorithm which takes in the results of measurements and outputs corrections to…
Ubiquity of Emergent Hebbian Dynamics in Regularized Learning
David Koplow, Tomaso Poggio, Liu Ziyin
Hebbian and anti-Hebbian plasticity are widely observed in the brain and are classically modeled as mechanistic, local homosynaptic rules stabilized by homeostatic constraints. Thi…
Thermodynamic Irreversibility of Training Algorithms
Liu Ziyin, Yuanjie Ren, Adam Levine +1
The training algorithms for AI systems all introduce far-from-equilibrium dynamical processes, and understanding the irreversibility of these algorithms is a fundamental step towar…
A universal compression theory for lottery ticket hypothesis and neural scaling laws
Hong-Yi Wang, Di Luo, Tomaso Poggio +2
When training large-scale models, the performance typically scales with the number of parameters and the dataset size according to a slow power law. A fundamental theoretical and p…