5 papers
A Heckler in the Hidden State: Correctness Signals in Diffusion Language Models
Angad Miglani, Samrath Singh Chadha, Kevin Li +1
Diffusion language models generate code by repeatedly updating a partially masked sequence. We ask whether their internal activations encode code correctness and whether that infor…
Decodable In-Context State and Model Output Across Training
Manas Venkata Sai Ravulapalli, Samrath Singh Chadha
Prior work established that a probe can decode an in-context binding on model errors and that probe-guided steering can repair some of them. We follow probe accuracy, model output,…
Quadratic bounds for uncompletable words and matrix mortality
Rahul Chandelkar, Samrath Singh Chadha, Samrath Chadha
Every finite nonempty incomplete uniquely decipherable code with maximum word length has an uncompletable word of length at most . The bound is independent of the numb…
In-Context Binding Capacity in Language Models
Manas Venkata Sai Ravulapalli, Samrath Singh Chadha
How many assignments can a language model recall before it loses track of which value belongs to which entity? We measure this limit using continuous recall curves for 12 models at…
Legible Failures: Detecting and Repairing In-Context Binding Errors
Manas Venkata Sai Ravulapalli, Samrath Singh Chadha, Abhinav M. Hari
A wrong answer does not show whether the model lacked the needed information or held it and failed to use it. On an entity-obligation binding task, a language model can emit an inc…