8 papers
Mitigating Over-Optimization in PRM-Guided Search in Mathematical Reasoning by Optimizing the Guide
Taejong Joo, Diego Klabjan
Process reward models (PRMs) provide dense step-level guidance for search-based reasoning, enabling inference-time compute to be allocated toward promising partial solutions. Howev…
Model-based Bootstrap of Controlled Markov Chains
Ziwei Su, Imon Banerjee, Diego Klabjan
We propose and analyze a model-based bootstrap for transition kernels in finite controlled Markov chains (CMCs) with possibly nonstationary or history-dependent control policies, a…
A Trainable Optimizer
Ruiqi Wang, Diego Klabjan
The concept of learning to optimize involves utilizing a trainable optimization strategy rather than relying on manually defined full gradient estimations such as ADAM. We present…
Central Limit Theorems for Transition Probabilities of Controlled Markov Chains
Ziwei Su, Imon Banerjee, Diego Klabjan
We develop a central limit theorem (CLT) for a non-parametric estimator of the transition matrices in controlled Markov chains (CMCs) with finite state-action spaces. Our results e…
Technical Debt in In-Context Learning: Diminishing Efficiency in Long Context
Taejong Joo, Diego Klabjan
Transformers have demonstrated remarkable in-context learning (ICL) capabilities, adapting to new tasks by simply conditioning on demonstrations without parameter updates. Compelli…
Differentiable Calibration of Inexact Stochastic Simulation Models via Kernel Score Minimization
Ziwei Su, Diego Klabjan
Stochastic simulation models are generative models that mimic complex systems to help with decision-making. The reliability of these models heavily depends on well-calibrated input…
Improving self-training under distribution shifts via anchored confidence with theoretical guarantees
Taejong Joo, Diego Klabjan
Self-training often falls short under distribution shifts due to an increased discrepancy between prediction confidence and actual accuracy. This typically necessitates computation…
IW-GAE: Importance Weighted Group Accuracy Estimation for Improved Calibration and Model Selection in Unsupervised Domain Adaptation
Taejong Joo, Diego Klabjan
Distribution shifts pose significant challenges for model calibration and model selection tasks in the unsupervised domain adaptation problem -- a scenario where the goal is to per…