2 papers
cs.LG2026
Dynamic Short Convolutions Improve Transformers
Oliver Sieberling, Bharat Runwal, Rameswar Panda +1
Transformers have become the dominant architecture for large language models, largely due to the scalability and flexibility of attention, feed-forward layers, residual connections…
cs.LG2026
PRISM: Demystifying Retention and Interaction in Mid-Training
Bharat Runwal, Ashish Agrawal, Anurag Roy +1
We present PRISM, a comprehensive empirical study of mid-training design choices for large language models. Through controlled experiments across seven base models spanning four fa…