12 citations · 41 across the 12 of their papers we have counts for
12 papers
A First-Principles Theory of Slow Thinking and Active Perception
Hongkang Yang, Zhi-Qin John Xu, Feiyu Xiong +1
As part of a series on first-principles modeling of cognitive functions, this paper attempts to provide a mathematical formulation of thinking and perception. It formally derives s…
Small Initialization Matters for Large Language Models
Liangkai Hang, Junjie Yao, Zhiyu Li +3
Large language models provide a tractable system for asking how intelligence itself emerges, rather than only how LLMs can be engineered. Although progress is usually attributed to…
MemOS: A Memory OS for AI System
Zhiyu Li, Chenyang Xi, Chunyu Li +36
Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI), yet their lack of well-defined memory management systems hinders the…
Adaptive Preconditioners Trigger Loss Spikes in Adam
Zhiwei Bai, Zhangchen Zhou, Jiajie Zhao +6
Loss spikes commonly emerge during neural network training with the Adam optimizer across diverse architectures and scales, yet their underlying mechanism remains elusive. While pr…
Scalable Complexity Control Facilitates Reasoning Ability of LLMs
Liangkai Hang, Junjie Yao, Zhiwei Bai +17
The reasoning ability of large language models (LLMs) has been rapidly advancing in recent years, attracting interest in more fundamental approaches that can reliably enhance their…
MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models
Zhiyu Li, Shichao Song, Hanyu Wang +19
Large Language Models (LLMs) have emerged as foundational infrastructure in the pursuit of Artificial General Intelligence (AGI). Despite their remarkable capabilities in language…
: Language Modeling with Explicit Memory
Hongkang Yang, Zehao Lin, Wenjin Wang +13
The training and inference of large language models (LLMs) are together a costly process that transports knowledge from raw data to meaningful computation. Inspired by the memory h…
A Mathematical Framework for Learning Probability Distributions
Hongkang Yang
The modeling of probability distributions, specifically generative modeling and density estimation, has become an immensely popular subject in recent years by virtue of its outstan…
Generalization Error of GAN from the Discriminator's Perspective
Hongkang Yang, Weinan E
The generative adversarial network (GAN) is a well-known model for learning high-dimensional distributions, but the mechanism for its generalization ability is not understood. In p…
Generalization and Memorization: The Bias Potential Model
Hongkang Yang, Weinan E
Models for learning probability distributions such as generative models and density estimators behave quite differently from models for learning functions. One example is found in…
Conditional Density Estimation, Latent Variable Discovery and Optimal Transport
Hongkang Yang, Esteban G. Tabak
A framework is proposed that addresses both conditional density estimation and latent variable discovery. The objective function maximizes explanation of variability in the data, a…
Clustering, factor discovery and optimal transport
Hongkang Yang, Esteban G. Tabak
The clustering problem, and more generally, latent factor discovery --or latent space inference-- is formulated in terms of the Wasserstein barycenter problem from optimal transpor…