4 citations · 4 across the 7 of their papers we have counts for
8 papers
CADOC: Cache-Aware Dynamic Object Context for Long-Horizon Agents
Junjie Yao, Zhangchen Zhou, Zhi-Qin John Xu
For a long-horizon agent, context is the bottleneck: the history is resent with every request, the window caps task length, and reasoning degrades as the history grows. Replacing s…
Context Staircase: Signature-Aligned Dynamics of Token Embeddings under Small Initialization
Junjie Yao, Liangkai Hang, Zhi-Qin John Xu
Token embeddings are the basic representational units that connect discrete tokens with continuous computation in language models. Although modern language models learn embeddings…
Small Initialization Matters for Large Language Models
Liangkai Hang, Junjie Yao, Zhiyu Li +3
Large language models provide a tractable system for asking how intelligence itself emerges, rather than only how LLMs can be engineered. Although progress is usually attributed to…
Probability Signature: Bridging Data Semantics and Embedding Structure in Language Models
Junjie Yao, Zhi-Qin John Xu
The embedding space of language models is widely believed to capture the semantic relationships; for instance, embeddings of digits often exhibit an ordered structure that correspo…
Scalable Complexity Control Facilitates Reasoning Ability of LLMs
Liangkai Hang, Junjie Yao, Zhiwei Bai +17
The reasoning ability of large language models (LLMs) has been rapidly advancing in recent years, attracting interest in more fundamental approaches that can reliably enhance their…
An Analysis for Reasoning Bias of Language Models with Small Initialization
Junjie Yao, Zhongwang Zhang, Zhi-Qin John Xu
Transformer-based Large Language Models (LLMs) have revolutionized Natural Language Processing by demonstrating exceptional performance across diverse tasks. This study investigate…
Anchor function: a type of benchmark functions for studying language models
Zhongwang Zhang, Zhiwei Wang, Junjie Yao +4
Understanding transformer-based language models is becoming increasingly crucial, particularly as they play pivotal roles in advancing towards artificial general intelligence. Howe…
Solving multiscale dynamical systems by deep learning
Junjie Yao, Yuxiao Yi, Liangkai Hang +5
Multiscale dynamical systems, modeled by high-dimensional stiff ordinary differential equations (ODEs) with wide-ranging characteristic timescales, arise across diverse fields of s…