Showing 2025Show all
3 papers · 1 filter
cs.LG2025
Transformers as Intrinsic Optimizers: Forward Inference through the Energy Principle
Ruifeng Ren, Sheng Ouyang, Huayi Tang +1
Attention-based Transformers have demonstrated strong adaptability across a wide range of tasks and have become the backbone of modern Large Language Models (LLMs). However, their…
cs.LG2025
Revisiting Transformers through the Lens of Low Entropy and Dynamic Sparsity
Ruifeng Ren, Yong Liu
Compression has been a critical lens to understand the success of Transformers. In the past, we have typically taken the target distribution as a criterion to evaluate a model's co…
cs.LG2025
Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning
Xinhao Yao, Ruifeng Ren, Yun Liao +2
Chain-of-Thought (CoT) training has markedly advanced the reasoning capabilities of large language models (LLMs), yet the mechanisms by which CoT training enhances generalization r…