1 citations · 1 across the 6 of their papers we have counts for
5 papers · 1 filter
Counting with Attention and LayerNorm: Norm Scaling and Representation Geometry
Ayyub Shaffy, Manoj Dhakal, Safal Shrestha +1
Counting is a simple task on which transformers can nevertheless fail at long sequence lengths. We study the cost of counting in a single attention layer, up to a maximum length $L…
On the Limits of Layer Pruning for Generative Reasoning in Large Language Models
Safal Shrestha, Anubhav Shrestha, Aadim Nepal +2
Recent work has shown that layer pruning can effectively compress large language models (LLMs) while retaining strong performance on classification benchmarks, often with little or…
Training Reasoning Models on Saturated Problems via Failure-Prefix Conditioning
Minwu Kim, Safal Shrestha, Anubhav Shrestha +1
As Reinforcement Learning with Verifiable Rewards (RLVR) substantially improves the reasoning abilities of large language models (LLMs), a new bottleneck emerges: more training pro…
Layer Importance for Mathematical Reasoning is Forged in Pre-Training and Invariant after Post-Training
Aadim Nepal, Safal Shrestha, Anubhav Shrestha +4
Large language models improve at math after instruction tuning, reinforcement learning, or knowledge distillation. We ask whether these gains come from major changes in the transfo…
Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges
Safal Shrestha, Minwu Kim, Keith Ross
Mathematical reasoning in Large Language Models (LLMs) is often evaluated using benchmarks with limited numerical ranges, failing to reflect real-world problem-solving across diver…