3 papers
cs.LG2026
CoDistill-GRPO: A Co-Distillation Recipe for Efficient Group Relative Policy Optimization
Soo Min Kwon, Ziteng Sun, Ananda Theertha Suresh +2
Group Relative Policy Optimization (GRPO) has emerged as a powerful algorithm for improving the reasoning capabilities of language models, but often fails to improve small models d…
cs.CL2024
Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines
Yuchen Li, Alexandre Kirchmeyer, Aashay Mehta +5
Autoregressive language models are the currently dominant paradigm for text generation, but they have some fundamental limitations that cannot be remedied by scale-for example inhe…
cs.CL2024
Exploring and Improving Drafts in Blockwise Parallel Decoding
Taehyeon Kim, Ananda Theertha Suresh, Kishore Papineni +3
Despite the remarkable strides made by autoregressive language models, their potential is often hampered by the slow inference speeds inherent in sequential token generation. Block…