16 papers
Low Perplexity is Repetition: A One-Dimensional Self-Conditioning Attractor in Continuous Diffusion LMs
Shuai Zhang, Zijie Chen, Hongliang He +2
Continuous diffusion language models such as ELF report record-low generative perplexity (Gen-PPL). We find a catch: these models repeat far more than human text, and Gen-PPL rewar…
Beyond Block Boundaries: Multi-Block Editing for Diffusion Large Language Models
Xingyu Mou, Zijin Huang, Tianze Zhang +5
Block diffusion is the dominant approach for scaling discrete diffusion language models (dLLMs), as fixed-size blocks preserve parallel decoding while keeping quadratic attention c…
Rewarding the Scientific Process: Process-Level Reward Modeling for Agentic Data Analysis
Zhisong Qiu, Shuofei Qiao, Kewei Xu +4
Process Reward Models (PRMs) have achieved remarkable success in augmenting the reasoning capabilities of Large Language Models (LLMs) within static domains such as mathematics. Ho…
StructMem: Structured Memory for Long-Horizon Behavior in LLMs
Buqiang Xu, Yijun Chen, Jizhan Fang +5
Long-term conversational agents need memory systems that capture relationships between events, not merely isolated facts, to support temporal reasoning and multi-hop question answe…
What Makes AI Research Replicable? Executable Knowledge Graphs as Scientific Knowledge Representations
Yujie Luo, Zhuoyun Yu, Xuehai Wang +6
Replicating AI research is a crucial yet challenging task for large language model (LLM) agents. Existing approaches often struggle to generate executable code, primarily due to in…
Can We Predict Before Executing Machine Learning Agents?
Jingsheng Zheng, Jintian Zhang, Yujie Luo +5
Autonomous machine learning agents have revolutionized scientific discovery, yet they remain constrained by a Generate-Execute-Feedback paradigm. Previous approaches suffer from a…