9 papers
On Language Drift during RLVR Post-Training
Michael Sullivan, Alexander Koller
Recent advances in LLM reasoning models---driven primarily by the paradigm of post-training via reinforcement learning with verifiable reward (RLVR)---have enabled them to accompli…
On the (In)effectiveness of AMR Augmentation for Large Language Models
Hoa Quynh Nhung Nguyen, Jacopo Staiano, Michael Sullivan
While Abstract Meaning Representation (AMR) has historically improved performance on a range of NLP tasks, the benefit---or lack thereof---of AMR augmentation for modern LLMs is th…
Accelerating Constrained Decoding with Token Space Compression
Michael Sullivan, Alexander Koller
To guarantee that an LLM's outputs conform to a specified structure, context-free grammar (CFG) decoding engines force the selection of next tokens to produce strings that conform…
AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing
Sarubi Thillainathan, Ji-Ung Lee, Michael Sullivan +1
The task of authorship style transfer involves rewriting text in the style of a target author while preserving the meaning of the original text. Existing style transfer methods tra…
Evaluating Spatiotemporal Consistency in Automatically Generated Sewing Instructions
Luisa Geiger, Mareike Hartmann, Michael Sullivan +1
In this paper, we propose a novel, automatic tree-based evaluation metric for LLM-generated step-by-step assembly instructions, that more accurately reflects spatiotemporal aspects…
GRPO is Secretly a Process Reward Model
Michael Sullivan, Alexander Koller
Process reward models (PRMs) allow for fine-grained credit assignment in reinforcement learning (RL), and seemingly contrast with outcome reward models (ORMs), which assign a singl…
Procedural Environment Generation for Tool-Use Agents
Michael Sullivan, Mareike Hartmann, Alexander Koller
Although the power of LLM tool-use agents has ignited a flurry of recent research in this area, the curation of tool-use training data remains an open problemespecially for onli…
Exploring Graph Representations of Logical Forms for Language Modeling
Michael Sullivan
We make the case for language models over logical forms (LFLMs), arguing that such models are more data-efficient than their textual counterparts. To that end, we introduce the Gra…
Playpen: An Environment for Exploring Learning Through Conversational Interaction
Nicola Horst, Davide Mazzaccara, Antonia Schmidt +13
Interaction between learner and feedback-giver has come into focus recently for post-training of Large Language Models (LLMs), through the use of reward models that judge the appro…