13 papers
Sample-Efficient Optimization over Generative Priors via Coarse Learnability
Pranjal Awasthi, Sreenivas Gollapudi, Ravi Kumar +1
We study zeroth-order optimization where solutions must minimize a cost while maintaining high probability under a complex generative prior (e.g., a parameterized mod…
Enhancing LLM Planning Capabilities through Intrinsic Self-Critique
Bernd Bohnet, Pierre-Alexandre Kamienny, Hanie Sedghi +7
We demonstrate an approach for LLMs to critique their \emph{own} answers with the goal of enhancing their performance that leads to significant improvements over established planni…
The Limits of Preference Data for Post-Training
Eric Zhao, Jessica Dai, Pranjal Awasthi
Recent progress in strengthening the capabilities of large language models has stemmed from applying reinforcement learning to domains with automatically verifiable outcomes. A key…
On Distributed Larger-Than-Memory Subset Selection With Pairwise Submodular Functions
Maximilian Böther, Abraham Sebastian, Pranjal Awasthi +2
Modern datasets span billions of samples, making training on all available data infeasible. Selecting a high quality subset helps in reducing training costs and enhancing model qua…
From Style to Facts: Mapping the Boundaries of Knowledge Injection with Finetuning
Eric Zhao, Pranjal Awasthi, Nika Haghtalab
Finetuning provides a scalable and cost-effective means of customizing language models for specific tasks or response styles, with greater reliability than prompting or in-context…
Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification
Eric Zhao, Pranjal Awasthi, Sreenivas Gollapudi
Sampling-based search, a simple paradigm for utilizing test-time compute, involves generating multiple candidate responses and selecting the best one -- typically by having models…