2 citations · 3 across the 7 of their papers we have counts for
8 papers
Limit Analysis for Symbolic Multi-step Reasoning Tasks with Information Propagation Rules Based on Transformers
Tian Qin, Yuhan Chen, Zhiwei Wang +1
Transformers are able to perform reasoning tasks, however the intrinsic mechanism remains widely open. In this paper we propose a set of information propagation rules based on Tran…
Achilles' Heel of Mamba: Essential difficulties of the Mamba architecture demonstrated by synthetic data
Tianyi Chen, Pengxiao Lin, Zhiwei Wang +1
State Space Models (SSMs) have emerged as promising alternatives to attention mechanisms, with the Mamba architecture demonstrating impressive performance and linear complexity for…
Scalable Complexity Control Facilitates Reasoning Ability of LLMs
Liangkai Hang, Junjie Yao, Zhiwei Bai +17
The reasoning ability of large language models (LLMs) has been rapidly advancing in recent years, attracting interest in more fundamental approaches that can reliably enhance their…
Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformers
Zhongwang Zhang, Pengxiao Lin, Zhiwei Wang +2
Transformers have demonstrated impressive capabilities across various tasks, yet their performance on compositional problems remains a subject of debate. In this study, we investig…
Loss Jump During Loss Switch in Solving PDEs with Neural Networks
Zhiwei Wang, Lulu Zhang, Zhongwang Zhang +1
Using neural networks to solve partial differential equations (PDEs) is gaining popularity as an alternative approach in the scientific computing community. Neural networks can int…
Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism
Zhiwei Wang, Yunji Wang, Zhongwang Zhang +7
Large language models have consistently struggled with complex reasoning tasks, such as mathematical problem-solving. Investigating the internal reasoning mechanisms of these model…
Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizing
Zhongwang Zhang, Pengxiao Lin, Zhiwei Wang +2
Transformers have shown impressive capabilities across various tasks, but their performance on compositional problems remains a topic of debate. In this work, we investigate the me…
Anchor function: a type of benchmark functions for studying language models
Zhongwang Zhang, Zhiwei Wang, Junjie Yao +4
Understanding transformer-based language models is becoming increasingly crucial, particularly as they play pivotal roles in advancing towards artificial general intelligence. Howe…