2 papers
cs.LG2026
Underscoring the Problem: Why Softpick Fails at Initialization
Aryan Sood, Jaikaran Singh, Ishaan Bansal
Softmax attention gives every token a nonzero weight, which in trained models concentrates into attention sinks and massive activations that widen the dynamic range low-precision i…
cs.CL2026
Lost in Interpolation: Why Predictive Feedback Fails in Diffusion Language Models
Lavanya Nigam, Ishaan Bansal, Aryan Sood +2
Soft-masking accelerates the convergence of Masked Diffusion Language Models (MDLMs). Existing formulations build this blend with linear interpolation (LERP) in the raw embedding s…