12 papers
One-Step Text Generation by Seq-Drifting
Jiayu Liu, Ruizhi Zhao, Tianyun Ji +3
Diffusion language models generate tokens in parallel, but still require multiple sampling steps to produce high-quality text. In this paper, we investigate whether a model can lea…
Explore Before Committing: Hypothesis-Guided Search for Deep Research Agents
Ruochen Zhou, Zhengyu Chen, Luan Zhang +3
Deep-research agents answer complex questions by interacting with search and browsing tools, yet they often search along a single evolving trajectory. Our trajectory-level analysis…
FaithformBench: Benchmarking Faithfulness of Mathematical Chain-of-Thought Autoformalisation
Rob Cornish, Iacopo Ghinassi, Po-Hung Yeh +7
Autoformalisation (AF) systems map natural language reasoning steps into formal statements in a proof assistant such as Lean. We consider how to assess the faithfulness of these sy…
SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?
Shiqi Chen, Jingze Gai, Ruochen Zhou +13
Real-world tool-using agents operate over long-horizon workflows with recurring structure and diverse demands, where effective behavior requires not only invoking atomic tools but…
Foundation of Intelligence: Review of Math Word Problems from Human Cognition Perspective
Zhenya Huang, Jiayu Liu, Xin Lin +6
Math word problem (MWP) serves as a fundamental research topic in artificial intelligence (AI) dating back to 1960s. This research aims to advance the reasoning abilities of AI by…
Enhancing Large Language Model Reasoning with Reward Models: An Analytical Survey
Qiyuan Liu, Hao Xu, Xuhong Chen +3
Reward models (RMs) play a critical role in enhancing the reasoning performance of LLMs. For example, they can provide training signals to finetune LLMs during reinforcement learni…
Deep Thinking by Markov Chain of Continuous Thoughts
Jiayu Liu, Zhenya Huang, Xuan Yang +6
Transformer-based models can perform complicated reasoning by generating reasoning paths token by token. While effective, this approach often requires generating thousands of token…
StochasTok: Improving Fine-Grained Subword Understanding in LLMs
Anya Sims, Thom Foster, Klara Kaleb +5
Subword-level understanding is integral to numerous tasks, including understanding multi-digit numbers, spelling mistakes, abbreviations, rhyming, and wordplay. Despite this, curre…
Meta-Learning Objectives for Preference Optimization
Carlo Alfano, Silvia Sapora, Jakob Nicolaus Foerster +2
Evaluating preference optimization (PO) algorithms on LLM alignment is a challenging task that presents prohibitive costs, noise, and several variables like model size and hyper-pa…
SymDiff: Equivariant Diffusion via Stochastic Symmetrisation
Leo Zhang, Kianoosh Ashouritaklimi, Yee Whye Teh +1
We propose SymDiff, a method for constructing equivariant diffusion models using the framework of stochastic symmetrisation. SymDiff resembles a learned data augmentation that is d…
EvIL: Evolution Strategies for Generalisable Imitation Learning
Silvia Sapora, Gokul Swamy, Chris Lu +2
Often times in imitation learning (IL), the environment we collect expert demonstrations in and the environment we want to deploy our learned policy in aren't exactly the same (e.g…
The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning
Anya Sims, Cong Lu, Jakob Foerster +1
Offline reinforcement learning aims to train agents from pre-collected datasets. However, this comes with the added challenge of estimating the value of behaviors not covered in th…