533 citations · 605 across the 21 of their papers we have counts for
6 papers · 1 filter
Neural Thermodynamic Laws for Large Language Model Training
Ziming Liu, Yizhou Liu, Jeff Gore +1
Beyond neural scaling laws, little is known about the laws underlying large language models (LLMs). We introduce Neural Thermodynamic Laws (NTL) -- a new framework that offers fres…
Symbolic Learning of Topological Bands in Photonic Crystals
Ali Ghorashi, Sachin Vaidya, Ziming Liu +4
Topological photonic crystals (PhCs) that support disorder-resistant modes, protected degeneracies, and robust transport have recently been explored for applications in waveguiding…
Do Two AI Scientists Agree?
Xinghong Fu, Ziming Liu, Max Tegmark
When two AI models are trained on the same scientific task, do they learn the same theory or two different theories? Throughout history of science, we have witnessed the rise and f…
Language Models Use Trigonometry to Do Addition
Subhash Kantamneni, Max Tegmark
Mathematical reasoning is an increasingly important indicator of large language model (LLM) capabilities, yet we lack understanding of how LLMs process even simple mathematical tas…
Harmonic Loss Trains Interpretable AI Models
David D. Baek, Ziming Liu, Riya Tyagi +1
In this paper, we introduce harmonic loss as an alternative supervisory signal for training neural networks and large language models (LLMs). Harmonic loss differs from standard cr…
Physics of Skill Learning
Ziming Liu, Yizhou Liu, Eric J. Michaud +2
We aim to understand physics of skill learning, i.e., how skills are learned in neural networks during training. We start by observing the Domino effect, i.e., skills are learned s…