1 citations · 1 across the 12 of their papers we have counts for
14 papers
Weight Decay and Neuron Condensation: A Three-Stage Analysis of Two-Layer ReLU Networks
Cheng Xu, Pengxiao Lin, Zhangchen Zhou +1
Weight decay is widely used as a regularization technique in neural network training, yet its role in neuron condensation (parameter direction alignment) remains unclear. Starting…
CADOC: Cache-Aware Dynamic Object Context for Long-Horizon Agents
Junjie Yao, Zhangchen Zhou, Zhi-Qin John Xu
For a long-horizon agent, context is the bottleneck: the history is resent with every request, the window caps task length, and reasoning degrades as the history grows. Replacing s…
Context Staircase: Signature-Aligned Dynamics of Token Embeddings under Small Initialization
Junjie Yao, Liangkai Hang, Zhi-Qin John Xu
Token embeddings are the basic representational units that connect discrete tokens with continuous computation in language models. Although modern language models learn embeddings…
Metis: Memory Foundation Model
Zeyu Zhang, Ziliang Guo, Yihang Sun +14
Recent advances in AI agents have increasingly internalized native capabilities into their underlying foundation models, giving rise to multimodal foundation models and large reaso…
Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay
Xiaolong Li, Zhangchen Zhou, Zhi-Qin John Xu
Most explanations of training instability focus on \emph{learning-rate criticality}, typically characterized by the Edge of Stability, beyond which optimization becomes unstable. W…
A First-Principles Theory of Slow Thinking and Active Perception
Hongkang Yang, Zhi-Qin John Xu, Feiyu Xiong +1
As part of a series on first-principles modeling of cognitive functions, this paper attempts to provide a mathematical formulation of thinking and perception. It formally derives s…
Small Initialization Matters for Large Language Models
Liangkai Hang, Junjie Yao, Zhiyu Li +3
Large language models provide a tractable system for asking how intelligence itself emerges, rather than only how LLMs can be engineered. Although progress is usually attributed to…
Towards Understanding Adam Convergence on Highly Degenerate Polynomials
Zhiwei Bai, Jiajie Zhao, Zhangchen Zhou +2
Adam is a widely used optimization algorithm in deep learning, yet the specific class of objective functions where it exhibits inherent advantages remains underexplored. Unlike pri…
Towards Reliable Pediatric Brain Tumor Segmentation: Task-Specific nnU-Net Enhancements
Xiaolong Li, Zhi-Qin John Xu, Yan Ren +2
Accurate segmentation of pediatric brain tumors in multi-parametric magnetic resonance imaging (mpMRI) is critical for diagnosis, treatment planning, and monitoring, yet faces uniq…
Neural Network Based Framework for Passive Intermodulation Cancellation in MIMO Systems
Xiaolong Li, Zhi-Qin John Xu, Peiting You +1
Passive intermodulation (PIM) has emerged as a critical source of self-interference in modern MIMO-OFDM systems, especially under the stringent requirements of 5G and beyond. Conve…
MemOS: A Memory OS for AI System
Zhiyu Li, Chenyang Xi, Chunyu Li +36
Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI), yet their lack of well-defined memory management systems hinders the…
Adaptive Preconditioners Trigger Loss Spikes in Adam
Zhiwei Bai, Zhangchen Zhou, Jiajie Zhao +6
Loss spikes commonly emerge during neural network training with the Adam optimizer across diverse architectures and scales, yet their underlying mechanism remains elusive. While pr…
Scalable Complexity Control Facilitates Reasoning Ability of LLMs
Liangkai Hang, Junjie Yao, Zhiwei Bai +17
The reasoning ability of large language models (LLMs) has been rapidly advancing in recent years, attracting interest in more fundamental approaches that can reliably enhance their…
MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models
Zhiyu Li, Shichao Song, Hanyu Wang +19
Large Language Models (LLMs) have emerged as foundational infrastructure in the pursuit of Artificial General Intelligence (AGI). Despite their remarkable capabilities in language…