Gaussian Process Aggregation for Root-Parallel Monte Carlo Tree Search with Continuous Actions
arXiv:2512.09727
The paper introduces a Gaussian Process regression method to aggregate statistics across parallel Monte Carlo Tree Search threads for continuous-action environments, showing improved performance over existing strategies with modest extra inference cost.
Abstract
Monte Carlo Tree Search is a cornerstone algorithm for online planning, and its root-parallel variant is widely used when wall clock time is limited but best performance is desired. In environments with continuous action spaces, how to best aggregate statistics from different threads is an important yet underexplored question. In this work, we introduce a method that uses Gaussian Process Regression to obtain value estimates for promising actions that were not trialed in the environment. We perform a systematic evaluation across 6 different domains, demonstrating that our approach outperforms existing aggregation strategies while requiring a modest increase in inference time.
To appear in the Reinforcement Learning Journal (RLJ) and to be presented at the Third Reinforcement Learning Conference (RLC 2026)