6 papers
Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
Xingyu Su, Jacob Helwig, Shubham Parashar +6
We study the transformation of autoregressive models (ARLMs) into diffusion language models (DLMs). Rather than pretraining from scratch, prior work replaces the causal attention i…
Learnability-Informed Fine-Tuning of Diffusion Language Models
Shubham Parashar, Atharv Chagi, Jacob Helwig +5
We aim to improve the reasoning capabilities of diffusion language models (DLMs). While SFT is a popular post-training recipe for autoregressive models, its use in DLMs faces chall…
Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning
Shubham Parashar, Shurui Gui, Xiner Li +8
We aim to improve the reasoning capabilities of language models via reinforcement learning (RL). Recent RL post-trained models like DeepSeek-R1 have demonstrated reasoning abilitie…
Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning
Tian Liu, Huixin Zhang, Shubham Parashar +1
Few-shot recognition (FSR) aims to train a classification model with only a few labeled examples of each concept concerned by a downstream task, where data annotation cost can be p…
Complex LLM Planning via Automated Heuristics Discovery
Hongyi Ling, Shubham Parashar, Sambhav Khurana +6
We consider enhancing large language models (LLMs) for complex planning tasks. While existing methods allow LLMs to explore intermediate steps to make plans, they either depend on…
Inference-Time Computations for LLM Reasoning and Planning: A Benchmark and Insights
Shubham Parashar, Blake Olson, Sambhav Khurana +4
We examine the reasoning and planning capabilities of large language models (LLMs) in solving complex tasks. Recent advances in inference-time techniques demonstrate the potential…