2 papers
cs.LG2026
Controllably Efficient Language Models
Jatin Prakash, Aahlad Puli, Rajesh Ranganath
The substantial inference costs of attention in transformers motivated the development of efficient sequence mixers: namely sparse and sliding window attention, convolutions and li…
cs.LG2025
KL-Regularized Reinforcement Learning is Designed to Mode Collapse
Anthony GX-Chen, Jatin Prakash, Jeff Guo +2
It is commonly believed that optimizing the reverse KL divergence results in "mode seeking", while optimizing forward KL results in "mass covering", with the latter being preferred…