1 citations · 1 across the 2 of their papers we have counts for
5 papers · 1 filter
RoPE attention is an exact forward-pass gradient step with softmax intact
Julie Huang, Maggie Chlon, Leon Chlon
We derive an exact gradient-step representation of the RoPE-softmax forward pass. For every deterministic RoPE-softmax attention head with arbitrary affine projection weights, we c…
Attention Deficits in Language Models: Causal Explanations for Procedural Hallucinations
Ahmed Karim, Fatima Sheaib, Zein Khamis +3
Large language models can follow complex procedures yet fail at a seemingly trivial final step: reporting a value they themselves computed moments earlier. We study this phenomenon…
Predictable Compression Failures: Order Sensitivity and Information Budgeting for Evidence-Grounded Binary Adjudication
Leon Chlon, Ahmed Karim, Maggie Chlon +1
Transformers used for evidence-grounded binary adjudication (e.g., support/refute, yes/no, or verifier-backed pass/fail decisions) can be sensitive to the order in which exchangeab…
LLMs are Bayesian, In Expectation, Not in Realization
Leon Chlon, Fatima Sheaib, Zein Khamis +3
Bayesian accounts of in-context learning face a direct objection: exact posterior predictives for exchangeable data are invariant to task-preserving order, yet transformers change…
Robust Multimodal Learning via Entropy-Gated Contrastive Fusion
Leon Chlon, Maggie Chlon, MarcAntonio M. Awada
Real-world multimodal systems routinely face missing-input scenarios, and in reality, robots lose audio in a factory or a clinical record omits lab tests at inference time. Standar…