9 papers
A Theory of LLM Information Susceptibility
Zhuo-Yang Song, Hua Xing Zhu
Large language models (LLMs) are increasingly deployed as optimization modules in agentic systems, yet the fundamental limits of such LLM-mediated improvement remain poorly underst…
Detailed balance in large language model-driven agents
Zhuo-Yang Song, Qing-Hong Cao, Ming-xing Luo +1
Large language model (LLM)-driven agents are emerging as a powerful new paradigm for solving complex problems. Despite the empirical success of these practices, a theoretical frame…
Where to Search: Measure the Prior-Structured Search Space of LLM Agents
Zhuo-Yang Song
The generate-filter-refine (iterative paradigm) based on large language models (LLMs) has achieved progress in reasoning, programming, and program discovery in AI+Science. However,…
Iterated Agent for Symbolic Regression
Zhuo-Yang Song, Zeyu Cai, Shutao Zhang +8
Symbolic regression (SR), the automated discovery of mathematical expressions from data, is a cornerstone of scientific inquiry. However, it is often hindered by the combinatorial…
Causal Attention with Lookahead Keys
Zhuoqing Song, Peng Sun, Huizhuo Yuan +1
In standard causal attention, each token's query, key, and value (QKV) are static and encode only preceding context. We introduce CAuSal aTtention with Lookahead kEys (CASTLE), an…
PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Models
Shi Qiu, Shaoyang Guo, Zhuo-Yang Song +51
Current benchmarks for evaluating the reasoning capabilities of Large Language Models (LLMs) face significant limitations: task oversimplification, data contamination, and flawed e…
Scalable Quantum State Preparation via Large-Language-Model-Driven Discovery
Qing-Hong Cao, Zong-Yue Hou, Ying-Ying Li +5
Efficient quantum state preparation remains a central challenge in first-principles quantum simulations of dynamics in quantum field theories, where the Hilbert space is intrinsica…
Bridging the Dimensional Chasm: Uncover Layer-wise Dimensional Reduction in Transformers through Token Correlation
Zhuo-Yang Song, Zeyu Li, Qing-Hong Cao +2
The geometric evolution of token representations in large language models (LLMs) presents a fundamental paradox: while human language inherently organizes semantic information in l…
Explainable AI-assisted Optimization for Feynman Integral Reduction
Zhuo-Yang Song, Tong-Zhi Yang, Qing-Hong Cao +2
We present a novel approach to optimizing the reduction of Feynman integrals using integration-by-parts identities. By developing a priority function through the FunSearch algorith…