7 citations · 13 across the 15 of their papers we have counts for
15 papers
Fast Weight Attention for Continual Learning
Yifan Zhang, Steve Ta, Jasper Zhang +8
Recurrent fast-weight memories and selective state-space models compress an expanding context into a fixed-size recurrent state, making the state transition an online learning rule…
Probing the Lack of Stable Internal Beliefs in LLMs
Yifan Luo, Kangping Xu, Yanzhen Lu +2
Persona-driven large language models (LLMs) require consistent behavioral tendencies across interactions to simulate human-like personality traits, such as persistence or reliabili…
Monadic Context Engineering
Yifan Zhang, Yang Yuan, Mengdi Wang +1
The proliferation of Large Language Models (LLMs) has catalyzed a shift towards autonomous agents capable of complex reasoning and tool use. However, current agent architectures ar…
Group Representational Position Encoding
Yifan Zhang, Zixiang Chen, Yifeng Liu +6
We present GRAPE (Group Representational Position Encoding), a unified framework for positional encoding based on group actions. GRAPE unifies two families of mechanisms: (i) multi…
Clarifying Before Reasoning: A Coq Prover with Structural Context
Yanzhen Lu, Hanbin Yang, Xiaodie Wang +6
In this work, we investigate whether improving task clarity can enhance reasoning ability of large language models, focusing on theorem proving in Coq. We introduce a concept-level…
Existing LLMs Are Not Self-Consistent For Simple Tasks
Zhenru Lin, Jiawen Tao, Yang Yuan +1
Large Language Models (LLMs) have grown increasingly powerful, yet ensuring their decisions remain transparent and trustworthy requires self-consistency -- no contradictions in the…
Towards Automated Formal Verification of Backend Systems with LLMs
Kangping Xu, Yifan Luo, Yang Yuan +1
Software testing plays a critical role in ensuring that systems behave as intended. However, existing automated testing approaches struggle to match the capabilities of human engin…
On the Design of KL-Regularized Policy Gradient Algorithms for LLM Reasoning
Yifan Zhang, Yifeng Liu, Huizhuo Yuan +3
Policy gradient algorithms have been successfully applied to enhance the reasoning capabilities of large language models (LLMs). KL regularization is ubiquitous, yet the design sur…
Hierarchical Attention Generates Better Proofs
Jianlong Chen, Chao Li, Yang Yuan +1
Large language models (LLMs) have shown promise in formal theorem proving, but their token-level processing often fails to capture the inherent hierarchical nature of mathematical…
ProofAug: Efficient Neural Theorem Proving via Fine-grained Proof Structure Analysis
Haoxiong Liu, Jiacheng Sun, Zhenguo Li +1
The synergy between deep learning models and traditional automation tools, such as built-in tactics of the proof assistant and off-the-shelf automated theorem provers, plays a cruc…
Tensor Product Attention Is All You Need
Yifan Zhang, Yifeng Liu, Huizhuo Yuan +4
Scaling language models to handle longer input sequences typically necessitates large key-value (KV) caches, resulting in substantial memory overhead during inference. In this pape…
On the Diagram of Thought
Yifan Zhang, Yang Yuan, Andrew Chi-Chih Yao
Large Language Models (LLMs) excel at many tasks but often falter on complex problems that require structured, multi-step reasoning. We introduce the Diagram of Thought (DoT), a fr…
Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Texts
Yifan Zhang, Yifan Luo, Yang Yuan +1
We present Autonomous Data Selection (AutoDS), a method that leverages base language models themselves as zero-shot "generative classifiers" to automatically curate high-quality ma…
Augmenting Math Word Problems via Iterative Question Composing
Haoxiong Liu, Yifan Zhang, Yifan Luo +1
Despite the advancements in large language models (LLMs) for mathematical reasoning, solving competition-level math problems remains a significant challenge, especially for open-so…
Meta Prompting for AI Systems
Yifan Zhang, Yang Yuan, Andrew Chi-Chih Yao
We introduce Meta Prompting (MP), a framework that emphasizes the formal structure of a task rather than content-specific worked examples. We give a categorical formalization in wh…