5 papers
Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled Capability Diagnosis
Xinhao Yao, Yuanzhuo Liu, Changhao Wang +6
Deep search is becoming a core capability of modern agent systems, yet it is typically evaluated solely based on end-to-end answer accuracy. This coupled evaluation paradigm entang…
Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning
Xinhao Yao, Ruifeng Ren, Yun Liao +2
Chain-of-Thought (CoT) training has markedly advanced the reasoning capabilities of large language models (LLMs), yet the mechanisms by which CoT training enhances generalization r…
Transformers as Intrinsic Optimizers: Forward Inference through the Energy Principle
Ruifeng Ren, Sheng Ouyang, Huayi Tang +1
Attention-based Transformers have demonstrated strong adaptability across a wide range of tasks and have become the backbone of modern Large Language Models (LLMs). However, their…
Exploring the Limitations of Mamba in COPY and CoT Reasoning
Ruifeng Ren, Zhicong Li, Yong Liu
Transformers have become the backbone of modern Large Language Models (LLMs); however, their inference overhead grows linearly with the sequence length, posing challenges for model…
Revisiting Transformers through the Lens of Low Entropy and Dynamic Sparsity
Ruifeng Ren, Yong Liu
Compression has been a critical lens to understand the success of Transformers. In the past, we have typically taken the target distribution as a criterion to evaluate a model's co…