Fine-Tuning Diffusion Language Models with Context Selection and Target Weighting

Loay Mualem, Lluís Pastor-Pérez, Vinh Tong +4

Supervised fine-tuning of discrete diffusion language models masks some response tokens and trains the model to recover their original values from the visible context. The masking…