Training-Free Character-Length Control in Summarization with Diffusion Language Models

Abstract

Summarization with a fixed character length is important for tasks such as headline and article preview generation, where text must fit an exact space. Autoregressive language models (LMs) struggle with precise character-length control: training-based methods require task-specific fine-tuning, while zero-shot prompting strategies achieve only approximate compliance. We propose DELTA (Diffusion Expected Length Targeting Algorithm), a training-free inference-time method for masked diffusion LMs that estimates expected character length at each denoising step and dynamically inserts or removes masked positions to steer toward a target length. DELTA requires no fine-tuning and is designed for off-the-shelf masked diffusion LMs. Across three summarization benchmarks, it reduces character-length errors by an order of magnitude compared to baselines while maintaining competitive summary quality in both automatic and human evaluation.

arXiv commentAccepted at EMNLP 2026 (Main Conference)

References in corpus (5)

• Summarization with Precise Length Control
• FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
• Dream 7B: Diffusion Large Language Models
• Controlling Summarization Length Through EOS Token Weighting
• $\rho$-$\text{EOS}$: Training-free Bidirectional Variable-Length Control for Masked Diffusion LLMs